DUDLEY KNOX LIBRARY 
NAVAL POSTGRADUATE SCHOOL 
MONTEREY CA 93943-5101 



NAVAL POSTGRADUATE SCHOOL 
Monterey, California 




THESIS 



THE POTENTIAL IMPACT OF HYPERSPECTRAL 
IMAGERY ON AMPHIBIOUS WARFARE PLANNING 


by 




Keith W. Maly 




December 1999 




Thesis Advisor: 


William K. Krebs 


Co-advisor: 


Richard C. Olsen 



Approved for public release; distribution is unlimited. 



REPORT DOCUMENTATION PAGE 


Form Approved 
OMBNo. 0704-0188 


Public reporting burden for this collection of information is estimated to average 1 hour per response, including the time for reviewing instruction, 
searching existing data sources, gathering and maintaining the data needed, and completing and reviewing the collection of information. Send 
comments regarding this burden estimate or any other aspect of this collection of information, including suggestions for reducing this burden, to 
Washington headquarters Services, Directorate for Information Operations and Reports, 1215 Jefferson Davis Highway, Suite 1204, Arlington, VA 


222UZ-4302, ana to tne Office of Management and Budget, PaperworK Reduction Project (0704-0188) Washington DC 20503. 


1. AGENCY USE ONLY (Leave blank) 


2. REPORT DATE 

December 1999 


3. REPORT T 

Master’s T 


YPE AND DATES COVERED 

’hesis 1 



4. TITLE AND SUBTITLE 

THE POTENTIAL IMPACT OF HYPERSPECTRAL IMAGERY ON AMPHIBIOUS 
WARFARE PLANNING 



6. AUTHOR(S) 

Maly, Keith W. 



7. PERFORMING ORGANIZATION NAME(S) AND ADDRESS(ES) 

Naval Postgraduate School 
Monterey, CA 93943-5000 



5. FUNDING NUMBERS 

N41 75699 WR97473 

PE: 060501 3N/F 1947 NELO 152 



8. PERFORMING ORGANIZATION 
REPORT NUMBER 



9. SPONSORING / MONITORING AGENCY NAME(S) AND ADDRESS(ES) 

Navy Technical Exploitation of National Capabilities (CNO-N6) 



10. SPONSORING /MONITORING 
AGENCY REPORT NUMBER 



11. SUPPLEMENTARY NOTES 

The views expressed in this thesis are those of the author and do not reflect the official policy or position of 
the Department of Defense or the U.S. Government. 



12a. DISTRIBUTION / AVAILABILITY STATEMENT 

Approved for public release; distribution is unlimited. 



12b. DISTRIBUTION CODE 



13. ABSTRACT (maximum 200 words) 

Military image analysts primarily use panchromatic and radar images to aid situational awareness in preparing a mission 
plan. Although analysts rely on these two formats, there are situations where these two sensors are unable to detect potential 
threats, i.e., buried mines. The Department of Defense has proposed using a hyperspectral sensor to detect threats that otherwise 
may not be detected by existing sensors. In order to determine the utility of hyperspectral imagery for mission planning, a task 
analysis was conducted at two Joint Intelligence Centers to measure image analysts’ preferences to infrared, radar, panchromatic, 
and hyperspectral imagery during an amphibious planning process. The results showed that the image analysts were most 
confident using panchromatic imagery for the majority of the planning tasks; however, the analysts exhibited uncertainty for 
other tasks, such as detecting buried mines. Further analysis showed that image analysts could reduce their uncertainty in 
detecting buried mines and producing bathymetric maps by using hyperspectral imagery. Although hyperspectral imagery 
reduced uncertainty during mission planning, operators report that this imagery is confusing. To integrate hyperspectral imagery 
in mission planning, image analysts must be trained to interpret a hyperspectral scene and understand how to exploit its’ spectral 
characteristics. 



14. SUBJECT TERMS 

Sensors, Visual Information Processing, Hyperspectral Imagery, Spectral Imagery, Image Interpretation, 
Amphibious Planning 


15. NUMBER OF 
PAGES 

149 


16. PRICE CODE 


17. SECURITY 

CLASSIFICATION OF REPORT 

Unclassified 


18. SECURITY CLASSIFICATION OF 
THIS PAGE 

Unclassified 


19. SECURITY CLASSIFI- CATION 
OF ABSTRACT 

Unclassified 


20. LIMITATION OF 
ABSTRACT 

UL 



NSN 7540-01-280-5500 



Standard Form 298 (Rev. 2-89) 
Prescribed by ANSI Std. 239-18 



ii 



Approved for public release; distribution is unlimited 



maw*. ^ ^ uior^MK Y 

■ l ^ 1 



THE POTENTIAL IMPACT OF HYPERSPECTRAL IMAGERY ON 
AMPHIBIOUS WARFARE PLANNING 



Keith W. Maly 

Lieutenant, United States Navy 
B.S., Rochester Institute of Technology, 1994 

Submitted in partial fulfillment of the 
requirements for the degree of 

MASTER OF SCIENCE IN SYTEMS TECHNOLOGY 
(SCIENTIFIC AND TECHNICAL INTELLIGENCE) 



from the 



NAVAL POSTGRADUATE SCHOOL 
December 1999 



ABSTRACT 
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Military image analysts primarily use panchromatic and radar images to aid situational 
awareness in preparing a mission plan. Although analysts rely on these two formats, 
there are situations where these two sensors are unable to detect potential threats, i.e., 
buried mines. The Department of Defense has proposed using a hyperspectral sensor to 
detect threats that otherwise may not be detected by existing sensors. In order to 
determine the utility of hyperspectral imagery for mission planning, a task analysis was 
conducted at two Joint Intelligence Centers to measure image analysts’ preferences to 
infrared, radar, panchromatic, and hyperspectral imagery during an amphibious planning 
process. The results showed that the image analysts were most confident using 
panchromatic imagery for the majority of the planning tasks; however, the analysts 
exhibited uncertainty for other tasks, such as detecting buried mines. Further analysis 
showed that image analysts could reduce their uncertainty in detecting buried mines and 
producing bathymetric maps by using hyperspectral imagery. Although hyperspectral 
imagery reduced uncertainty during mission planning, operators report that this imagery 
is confusing. .To integrate hyperspectral imagery in mission planning, image analysts 
must be trained to interpret a hyperspectral scene and understand how to exploit its’ 
spectral characteristics. 
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INTRODUCTION 



Military planners transform doctrine, objectives, and capabilities into the 
commander’s concept for decisive battlefield action. One of the most fundamental tasks 
that planners perform is to determine the relative location and possible effects of certain 
essential elements: mission objective, friendly capabilities, battlefield characteristics, 
enemy responses, and weather. The planner’s measure of effectiveness is the 
commander’s approval of a proposed plan. Planning missions for naval forces can be 
difficult in some situations because the objective area can be located a hemisphere away 
from the ship and available planning resources might be limited. In order to build the 
comprehensive plan that the commander will approve and execute, the planner relies on 
multiple sources of information. These sources might include intelligence in the form of 
satellite or reconnaissance imagery, historical data, reports from personnel inside the 
operating area, and enemy electronic signals. 

If a mission planner needs information extracted from a remotely sensed image 
such as panchromatic, infrared, and multispectral satellite imagery (MSI), he or she will 
rely on the skills of the image analyst. An experienced image analyst exploits the 
strengths and weaknesses of each media format to gain insight about the operating area, 
providing valuable information that an untrained person could not. However, this process 
has limitations. Inaccuracy and incomplete information in these formats decrease the 
amount of available intelligence that can be used to portray the battlefield accurately. For 
example, maps may not display all relevant features. Visible (panchromatic) sensors 
require clear visibility during the daytime, and may not provide useful data. Infrared 
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sensors give information on thermal differences between objects within a scene, but are 
still subject to interference from clouds, and may not distinguish essential elements of 
information. Traditional imagery may not answer some important questions. If the 
analyst is fortunate, other intelligence sources are available to compensate for these 
problems, though with lower confidence. The image analyst or mission planner must 
then determine which information source is most accurate if they happen to contradict. 
Interpretation problems and errors create intelligence gaps that introduce uncertainty into 
the planning process. 

For example, a military unit planning an amphibious assault is concerned with 
how fast friendly forces can ingress inland. Enemy defenses and soil trafficability are 
just two factors that may hinder this movement. Even with high image quality, the 
analyst may not detect well-camouflaged defenses or ascertain soil type on panchromatic 
imager,'. However, infrared (IR) sensors can distinguish camouflage from natural 
vegetation and MSI can be used to classify soils. On the other hand, mission planners 
might mistrust information from sensors such as infrared or MSI because these formats 
often do not provide photograph-like images. If multiple information sources or trained 
interpreters are not available, there will be more intelligence gaps. The planner must still 
propose a concept of operation, but he or she will be less confident in the overall plan. 
Such a plan may result in the planner unwittingly rejecting the best available area for 
landing and selecting a higher risk area instead. 

In most cases, the provided intelligence is adequate; however, there are situations 
when the available intelligence resources fail to adequately depict the battlefield. 
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Alternative resources should be evaluated for possible integration as operational 
capabilities, provided they offer the planner the intelligence necessary to gain the 
commander’s confidence in the mission plan. The Naval Tactical Exploitation of 
National Capabilities (TENCAP) office is investigating hyperspectral-imaging 
technology (HSI) to determine if it meets the above criteria (Navy TENCAP, 1998). HSI 
captures spatial information about the Earth in the format of hundreds of discrete 
electromagnetic energy wavelength bands. Proponents of hyperspectral imaging believe 
that applying non-literal image interpretation methods on the HSI data set can fill some of 
these intelligence gaps (Jensen, 1999). Their confidence is based in part on the 
technology’s successful commercial applications in a variety of environmental 
monitoring functions (Lewotsky, 1994). 

The purpose of this thesis is to determine whether HSI can increase the image 
analyst’s confidence level during image interpretation. It is hypothesized that HSI will 
provide unique information that other formats cannot provide and/or in areas where the 
analyst is currently uncertain. By integrating this information into the existing 
knowledge base about an area of interest, the mission planner will have a more complete 
battlefield picture. Therefore, a better mission plan is developed that the commander has 
more confidence in. 
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II. AMPHIBIOUS WARFARE 



The following evaluation of HSI utility is conducted in the context of Amphibious 
Warfare. Amphibious Warfare is the art of maneuvering military forces on a seaward 
flank to achieve various higher-level objectives (JP 3-02, 1992). Coalition forces 
designed the amphibious feint during the Gulf War in order to keep the Iraqis’ attention 
focused partially seaward while ground forces outmaneuvered the Iraqis’ forces in the 
deserts of Kuwait and Southern Iraq (Bernard and Trainor, 1995). In contrast. General 
MacArthur landed US Marines at Inchon in 1951 to open a second front against North 
Korean forces attacking the allies at Pusan (Gatchel, 1996). 

Amphibious operations differ from conventional ground operations in a number 
of ways. In a conventional ground operation, the physical distance that separates 
opponents is relatively small, while the distance between an amphibious landing force 
and its enemy is typically greater than fifty miles. Also, the conventional ground 
commander has combat power such as infantry, tanks, artillery, and possibly aircraft on 
the battlefield ready to conduct the operation. One of the greatest challenges in planning 
amphibious operations is that the commander cannot depend on any combat power ashore 
to be immediately available at the start of the landing. All the combat power must be 
ferried from the ship to the shore in the face of a non-cooperative enemy before ground 
combat designed to secure the objective can commence (FMFM 3-21, 1991). Finally, the 
landing forces’ planning elements are scattered among the ships comprising the 
Amphibious Ready Group (ARG). This preventive measure minimizes the complete loss 
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of any single element should a ship be lost or damaged. However, a side effect of this 
dispersal is that it complicates planning efforts and information dissemination. 

The planning phase described below is the first major step in an amphibious 
operation. Personnel from various warfare specialties such as infantry, armor, artillery, 
logistics, communications, and others all have valuable roles to play in planning the 
amphibious operation. However, this thesis will focus on the impact that intelligence 
and, more specifically, imagery intelligence (IMINT) has on the planning process. 

A. PLANNING APPROACH 

The commander’s staff cannot develop an acceptable plan until the commander 
receives an unambiguous estimate of the operating area and the enemy capabilities and 
courses of action. Initial planning estimates are produced to meet this need, thus 
establishing baseline knowledge about the operating area. Developing this baseline from 
a warship is no simple task. The commander’s intelligence officer begins this task with 
only the information directly available aboard ship. Organic reconnaissance assets such 
as unmanned aerial vehicles (UAV) or aircraft must be within the tactical range of the 
operating area in order to commence intelligence collection. Therefore, it is quite 
possible that during the transit to the objective area embarked intelligence personnel are 
almost completely reliant on external agencies such as theater-level Joint Intelligence 
Centers (JIC) to keep them apprised of the enemy situation. Two estimates of concern 
are the Intelligence Estimate and Intelligence Preparation of the Battlefield (JP 3-02, 
1992; FMFM3-21, 1991). 
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1 . 



Intelligence Estimate 



The Intelligence Estimate is developed to assist the commander in preparing his 
situation estimate, which is needed before the mission can begin to be planned in earnest 
by the commander’s staff. One of the functions of the intelligence estimate is to reduce 
the uncertainty of the physical environment. The five-part textual document first 
concisely reviews the assigned mission. Then it describes the Area of Operations in 
terms of military geography, hydrographic data, and climatic effects on friendly 
operations. Next, enemy military capabilities and vulnerabilities are thoroughly 
reviewed. Finally, the estimate concludes by integrating these individual segments into a 
final assessment that includes the possible enemy courses of action based on their relative 
probability and enemy vulnerabilities that friendly forces can exploit (FMFM 3-21, 
1991). 



2. Intelligence Preparation of the Battlefield (IPB) 

In contrast to the intelligence estimate, IPB is a graphical method of expressing 
the information contained in the intelligence estimate. It is a circular approach that 
integrates information about the terrain, weather, enemy, and one’s own mission into a 
comprehensive picture that allows the commander to continually evaluate enemy course 
of action, enemy vulnerabilities, and one’s own courses of action. After defining the 
operating area, the process evaluates the terrain using military topographical maps and 
imagery. The goal of terrain evaluation is to determine how terrain affects friendly and 
enemy abilities to shoot, move, and communicate. Weather effects on mobility and 
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visibility are then integrated with terrain information. To determine what the enemy is 
capable of actually performing, the intelligence staff evaluates and merges threat 
information with terrain, weather, and doctrine. The final products from each iteration of 
this process are battlefield “snap shots,” lists of possible targets, and commander decision 
points (FMFM3-21, 1991). 

3. Imagery Interpretation in Support of Amphibious Operations 

Image analysts must complete several types of tasks— classification, enumeration, 
mensuration, and delineation —individually or in combination to support intelligence 
production. Classification is the assignment of objects, features, or areas to classes based 
on appearance. An analyst may detect, recognize, or identify objects in a scene to 
complete classification. When performing enumeration the analyst detects and classifies 
discrete items in a scene so that they may be listed or counted. Mensuration is the 
measurement of object dimensions either to describe an area or to aid in classification. 
Finally, delineation is separation of regions within a scene based on criteria that, in this 
case, support the basic decisions (Campbell, 1996). 

The image analyst uses elements of recognition to aid in completing the tasks 
described above. The elements of recognition are shape, size, tone, texture, pattern 
shadow, site, and association. The first five self-explanatory elements highlight basic 
aspects of objects contained in a scene. Site and association capture secondary 
information by building on the first five elements (Campbell, 1996). 
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The image analyst and the mission planner will probably not have face-to-face 
contact. Therefore, the analyst must have an idea of what information the planner is 
interested in, in order to focus the interpretation effort towards fulfilling those needs. 
These pieces of information are called Essential Elements of Information, which are 
listed as Generic Intelligence Requirements (GIRH, 1991). 



B. BASIC DECISIONS 

The basic decisions are twelve fundamental decision areas that affect the 
amphibious operation. They are partially based on the intelligence provided in the 
Intelligence Estimate and the IPB. The decision points are usually made in the order 
listed below; however, a decision point may be suspended while waiting for more 
information (JP 3-02, 1992). The basic decisions are: 

• select the amphibious task force (ATF) general course of action 

• select ATF objectives 

• determine landing force (LF) mission 

• designate landing sites 

• determine LF objective 

• determine beachheads 

• select landing areas 

• formulate LF concept of operations 

• select landing beaches 

• select helicopter landing zones 

• select fixed wing aircraft drop zones 

• select d-day and h-hour 
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The result of the basic decisions is a task force plan. From this point, units 
subordinate to the task force may begin planning their operations based on the basic 
decisions (JP 3-02, 1992). In summary, regardless of the force size placed ashore, a 
logically conceived plan improves the chances of meeting the objectives set forth by the 
issuing commander, and planning aids are at the heart of this effort. 
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III. VISUAL COGNITION 



Traditional imagery analysis focuses on literal processing. This differs from the 
general approach to analyzing spectral imagery (MSI, HSI), and it is important to 
understand the contrast in the two approaches. The analytic process starts with a pictorial 
representation of a geographic region, or scene. Next, the analyst breaks the scene into 
specific parts. These parts are recognizable to most individuals who view the scene, but 
because of their additional training and experience, analysts can provide knowledge about 
those parts that untrained personnel cannot. The mission planner then uses the 
knowledge drawn from these parts to support the mission plan. 

In psychological terms, the analyst is using perception to recognize and interpret 
visual stimuli. Photons are the stimuli that the human visual system collects. Through a 
complex process, they generate a perceptual image that embodies the properties the 
photons carry (Howes, 1990). Matlin (1994) states that the act of perceiving visual 
objects combines aspects of both the stimuli and the observer’s internal processes. 

One approach to filtering the immense body of knowledge associated with visual 
cognition is the information processing approach, which focuses on the psychological 
processes associated with transforming the stimulus into information. Within the 
information processing approach, a means of differentiating the model types is to 
determine which of the aspects that Matlin (1994) describes, stimulus or internal 
processes, has more emphasis placed on them. Bottom-up models place more emphasis 
on the stimulus, while top-down models place more emphasis on the observer’s inner 
processes. General descriptions and models specific to each of these methods are 
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reviewed in the next two subsections. Figure 1 is gun camera footage of a tank taken 
during Operation Allied Force, which is used to illustrate important points associated 
with the different approaches in the hypothetical task of tank detection (FAS, 1999). 




Figure 1. Gun Camera Footage of a Tank (FAS, 1999) 

A. BOTTOM-UP PROCESSING 

Bottom-up processing models assert that pattern recognition activities are based 
solely on the nature of the incoming stimulus (Chase, 1986). The human visual system 
identifies properties of the stimulus and converts them into perceptual information that 
then makes contact with processes and structures in memory (Howes, 1990). Template, 
feature and computational theories are all examples of bottom-up processing models. In 
general, bottom-up theories would assume that imagery interpretation activities 
associated with tank detection are based solely on the analyst's ability to identify stimuli 
associated with tanks that are present in the scene. 
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1 . 



Template Theory 



Template-based theories assume that pattern recognition activities match the 
stimulus to specific object templates in memory (Chase, 1986). Templates in memory 
are based on objects that have already been encountered. Each template captures unique 
shape, size, and orientation details for a unique object. A new template is created if any 
of those characteristics differ from existing templates. Objects are recognized when an 
exact match occurs between stimulus and template (Matlin, 1994). An analyst in the 
example posed would examine the scene looking for stimuli that exactly matches a “tank 
template.” 

While these theories work well for explaining how simple, standardized objects in 
uncluttered scenes are identified, template theories are unable to explain how humans 
pattern recognize objects in complex environments (Howes, 1990). Exact template 
matching to unprocessed sensory input implies that an infinite number of templates for 
each type of tank are stored in memory because the initial size and orientation are 
unknown. As stated, template theories falsely assert that an analyst could not classify a 
tank if only part of the tank is seen. Even with processing necessary to standardize the 
input before matching, an infinite number of standardized templates would be required 
for all the possible objects a human could encounter visually (Chase, 1986). 

2. Feature Theory 

Feature theory differs from template theory in that it focuses on analyzing and 
representing sensory information in primitive information units called features vice 
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templates (Chase, 1986; Matlin, 1990). This theory bypasses the problems associated 
with template theory because object identification and discrimination between objects 
occurs through the comparison and contrast of distinctive features (Matlin, 1994). Tanks 
can be detected in feature theory because tanks have two treads and a turret that the 
analyst can identify. 

The major problem with feature theory is that it does not address the arrangement 
of features (Howes, 1990). An analyst who detects two treads and one turret by feature 
theory has detected a tank. However, if the turret is not centered between the treads, then 
they have detected something other than a tank. Feature theory makes no accommodation 
for this problem. 

3. Computational Theory 

Computational theories attempt to describe computer-based activities that 
replicate cognitive tasks that humans perform. The major computational model is a 
multi-stage model that begins with a two-dimensional array of numbers that correspond 
to the incoming photons’ intensity (Matlin, 1994). Numerical operations on the array 
during each stage involve different and increasingly complex computations to convert the 
array into primitive units by edge detection, then two-dimensional surfaces, and finally 
three-dimensional objects that are compared to memory (Matlin, 1994). Computational 
theory is incomplete at the final levels because the transition from two- or three- 
dimensional surfaces to the final level of perceptual data is not well understood. Later 
work uses this theory to provide the basis for three-dimensional feature detection theories 
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(Matlin, 1994). In computation theory, tank identification occurs because an analyst is 
able to first identify the tank’s edges and, through increasingly complex computations on 
the visual field, build two-dimensional and then three-dimensional surfaces that lead to 
the tank identification. 

B. TOP-DOWN PROCESSING 

Top-down processing differs from bottom-up processing in that it accounts for 
factors that are not present in the physical stimulus, such as the higher-level processes 
brought to bear in perception. Top-down processing makes it possible to identify 
distorted, ambiguous, and missing stimuli that reduce reaction times for pattern 
recognition. Context and knowledge are considered major influences in top-down 
processing (Howes, 1990). 

1. Context Effects 

Context provides expectations concerning the nature of stimuli that surround a 
target of interest (Howes, 1 990). The observer processes the least ambiguous information 
in a scene first, because it provides clues toward classifying the ambiguous stimuli 
(Matlin, 1994). If the analyst is conducting a wide area search for tanks, context can help 
focus the search in two different ways. First, it helps eliminate areas from the search 
space. For example, tanks cannot operate in certain terrain types, so the analyst does not 
need to look in this terrain for tanks. Second, it points the analyst toward particular areas 
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by using other intelligence sources. The advantage that context provides is that it 
prevents the analyst from having to search the entire scene for tanks. 

In remote sensor image interpretation, understanding context is particularly 
important because the raw information provided by the sensor can be in a dramatically 
different format than that usually experienced by humans. Air- or space-borne sensors 
collect their data from an overhead perspective. In addition, they can exploit 
electromagnetic radiation outside the region that the human eye senses (Campbell, 
1996). However, the analyst understands what characteristics the remote sensor captures 
and uses that information to aid in interpretation. In a daytime infrared image, a tank will 
probably be warmer than the surrounding environment, so the analyst might focus on the 
brighter sections of a scene for tank characteristics. 

2. Knowledge Effects 

Knowledge gives the analyst familiarity with the objects in a scene (Matlin, 
1994). One form of general knowledge that humans use unconsciously is perceptual 
constancy. This is a higher-order capacity that stabilizes the environment’s appearance 
across different domains even though the retinal image is continually changing. From 
this stable environment, it is then possible to detect and identify objects. Shape, size, and 
color are three domains in which perceptual constancy operates (Matlin, 1994; Sekuler, 
1994). In a tank search, the analyst is able to identify a tank from almost any aspect 
because of shape-constancy. Similarly, if the analyst zooms in on a portion of a scene 
and sees a tank, he or she knows from size constancy that the tank’s physical size did not 
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actually change. Finally, displaying images in gray scale bypasses the need to establish 
or maintain color-constancy. This is done in part because some sensors such as infrared 
and radar collect their data outside the range of the human eye. Therefore, using color in 
this instance can confuse the analyst because the information conveyed by color does not 
have an equivalent in real life. 

Trained experts can also use subject matter expertise to organize information in 
ways that facilitate its storage and retrieval (Chase, 1986). Military image analysts have 
specialized knowledge about both the remote sensor and an enemy in a particular 
geographic location (Campbell, 1996). If the analyst knows that an armor division is 
deployed to a region, they can apply knowledge about both the order of battle and troop 
disposition to the task of finding tanks. 

3. Stages Model 

The primary model that psychologists use to explain top-down processing is the 
stages model (Neisser, 1967), which is a discrete, sequential model for visual information 
processing. The lower-level analyses or preattentive processes rapidly test the visual 
field for simple target characteristics such as color, shape, and orientation in parallel 
(Neisser, 1967; Chase, 1986; Matlin, 1994). The preattentive mechanisms automatically 
register features. Targets that are discemable by simple criteria will appear to “pop out” 
of a scene no matter how many items are in the display (Chase, 1986; Matlin, 1994). The 
higher-level or focal attentive processing occurs when cognitive resources are applied to 
just a portion of the visual field (Chase, 1986). Focal attentive processes can use the 
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features registered in the preattentive phase to perform operations including grouping, 
target identification based on multiple criteria, and confirming a target’s absence (Chase, 
1986). Ultimately, focal attentive processes take advantage of context and knowledge to 
build an appropriate object that may be compared with the existing knowledge base in 
memory in order to make a judgment (Neisser, 1967). 

Later elaboration of the stages model includes setting a threshold value for 
identification of given stimuli (Howes, 1990). This is significant because only partial 
processing is required before making a judgment call. In this case, an analyst would only 
need to see part of a camouflaged tank to detect it. 

C. COGNITIVE MAPS 

After the analyst has processed a scene, the task is not yet complete; something 
must be done with the analysis. In military applications, for example, the location 
information accompanying the processed information is as important as the detected 
objects. The analyst builds internal abstract diagrams called cognitive maps to represent 
the result of that interpretation (Chase, 1986). Storage and processing errors are two 
issues associated with cognitive maps. 

1. Storage 

The method of storing data affects how quickly it can be used. Two such methods 
are hierarchical storage and route-based storage. Hierarchical storage does not attempt to 
describe the spatial relationships between all the objects in a scene, but rather describes 
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and stores relationships by subregion. Relationships between subregions are then stored, 
and so on up the hierarchy (Chase, 1 986). A military example of this hierarchical storage 
involves the display and representation of ground units. On a small-scale (large area) 
map, perhaps only higher-level military units are of interest, so only unit strengths greater 
than a brigade are plotted. However, the brigade’s alignment in the maneuver space is 
based on functions such as movement, offense, or defense. This function determines how 
the smaller-sized units (regiments, battalions, etc.) that comprise the brigade are located 
in this area. This hierarchical relationship can exist on a sliding scale for the ground 
units. 

Route-based storage involves organizing the movement as a series of connected 
vectors. Significant information is then stored relative to the route vector (Chase, 1986). 
A military example of this type of storage is a pilot’s flight plan. The pilot completes a 
mission by flying the aircraft along a series of connected points (flight path). Along the 
flight path, the pilot completes certain actions that are related to his mission, and thus 
stores information such as enemy units, using the flight path as means of ordering data. 

2. Errors 

There are usually errors associated with information encoding and application, 
because of the abstract nature of information storage. Alignment errors describe 
difference in displacement between the stored and actual locations of spatial information. 
The direction of the error usually tends toward a cardinal reference point. Internally 
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stored information that is stored in an absolute reference frame is of lesser value until it is 



rotated toward the user’s reference frame (Chase, 1986). 

The analysis conducted below suggests that the top-down model for visual 
cognition is the better model for the interpretation process as observed in this study 
because much of the knowledge that the analyst uses cannot be explained in a purely 
bottom-up paradigm. 
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IV. PLANNING AIDS 



The intelligence estimate can convey a large amount of information about the 
operating area. The planner will also use pictorial representations of the operating area 
such as topographical maps and remotely sensed images in conjunction with the 
intelligence estimate to plan the mission. The two types of planning aids, textual 
documents and pictures, together have a synergistic effect because the planner uses the 
image as a way to manipulate knowledge in memory (Chase, 1986). 

The formats most commonly used by planners and interpreters are topographical 
maps, black and white images that contain information from a single spectral band, and 
possibly color images containing information about two or more spectral bands. This 
section describes each of these formats and characterizes the strengths associated with the 
format-how well it describes the operating area — and its challenges or limitations in 
interpretation. The level of detail contained in the format descriptions provides the reader 
with an understanding of how the analyst interprets the data presented. Part of the 
challenge described later is how to integrate MSI/HSI into the planning process and the 
product, the packet. 

The final product of the intelligence production process is a packet that is 
delivered to the amphibious force. Such packets generally are composed of maps and 
small image chips for targets and areas of interest. Should the force be called on to plan a 
mission, they will commence preparation with these products. While certain 
multispectral images have been included in these packages in the past, hyperspectral data 
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have not, and one of the challenges of this thesis is to determine how to integrate this 
information into the package. 

A. MAPS 

Image analysts and planners both use maps to orient themselves to the area of 
interest. The map scales primarily used in planning are topographic (1:50,000) and Joint 
Operations Graphics (1:250,000). These scales represent the best balance between 
coverage and specificity (FM 21-26, 1993). Maps are the primary medium that planners 
use to prepare missions because in many instances the map provides the planner most of 
the information necessary to plan and execute a mission. Analysts also use maps, but 
they use the map more as reference material, such as comparing the features in an image 
to the map or vice versa or for plotting the approximate location of a target of interest. 

Military topographical maps are graphical representations of a portion of the 
Earth’s surface. Map-making starts with a mathematical model to chart the surface as 
viewed from above. The model ensures that the spatial displacement between objects is 
accurate relative to geo-coordinates within a given specification for a particular scale. 
Additional data is added to the map based on the information contained in aerial or 
satellite photographs and ground surveys (Gervais, 1999). Ground features are 
represented using standardized colors, symbols, and labels. Figures 2 and 3 as well as 
Table 1 provide examples of how each type of data is encoded. Only static information is 
printed on a map, since revision typically does not occur more often than once every 
several years. Therefore, the placement of dynamic targets such as mobile troop 
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positions would be irrelevant due to the long time lag between map development and use 
(FM 21-26, 1993; AFSC1N151, 1995). The chart’s legend provides example symbols on 
an uncluttered background with a concise textual description of the symbol’s meaning, 
such as the one provided in Figure 3. However, elevation and background coloring are 
two areas where information encoding is not straightforward. 




Figure 2. 1:50,000 Topographical Map of Kontum, Vietnam (NIMA, 1995) 
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Figure 3. Legend Examples (US Department of the Interior, 1994) 



Color 


Meaning 


black 


man-made cultural features (roads, buildings, surveyed elevation spots) 


red-brown 


natural cultural features, relief features, non-surveyed spot elevations 


blue 


hydrographic data, lakes, swamps, rivers, and drainage 


green 


vegetation of military significance 


brown 


relief features and elevation on older maps 


red 


cultural features, populated areas, main roads, and boundaries 


purple 


information related to safety of flight 



Table 1. Military Map Color Coding Schemes (FM 21-26, 1993) 



1. Elevation 

The use of contour lines is the most common method for displaying elevation on a 
map. A contour line connects a series of points that are at the same elevation. The line 
also describes the terrain’s shape at a particular elevation. Contour lines are drawn based 
on a standard change in elevation that varies with map scale. Terrain features are more 
precisely described by adding more contour lines to the map. Numerical values printed 
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occasionally near a contour line provide the reader a reference elevation to calculate other 
elevations. Enclosed contour lines that are inside other enclosed lines are of a higher 
elevation. Relative spacing between lines depicts the surface gradient; shallower 
gradients have more space between line pairs (FM 21-26, 1993; AFSC1N151, 1995). 
Figure 4 shows examples of contour lines for different terrain features. 

Elevation data is also available in a digital form called Digital Terrain Elevation 
Data (DTED). DTED is a matrix of terrain elevation values for a region of the world, 
defined by one-degree square cells. There are different levels of DTED quality. For 
example, a DTED Level 0 cell has 1 000-meter lateral separation between matrix values. 
Additional software such as NIMAMUSE are required to read and display DTED data 
(NIMA, 1998). Table 2 summarizes the accuracy levels for DTED Levels 0 through 5 
(FAS, 1997). 




Figure 4. Terrain Feature Description Using Contour Lines (FM 21-26, 1993) 
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DTED Level 


Post Spacing 


0 


100 m 


1 


100 m 


2 


30 m 




10 m 


4 


3 m 


5 


1 m 



Table 2. DTED Level Post Spacings (FAS, 1997) 



All forces are interested in the enemy’s geographic location relative to 
surrounding terrain because it is an indication of what the enemy is prepared to do, but air 
and land forces view the terrain map differently when considering their own maneuver. 
Mechanized ground forces prefer wide-open terrain because it gives them the ability to 
maneuver. Terrain with prominent elevation features provides mobile forces such as 
aircraft or small infantry units with landmarks to navigate by and can offer cover, 
concealment, and observation areas. 

2. Color 

Standardized coloring schemes are often used to encode background information. 
Table 1 provides a list of colors and their corresponding encoded features on military 
charts (FM 21-26, 1993). Differences in color are adequate to identify major differences 
between areas such as the difference between land and water or between desert and 
forest. A problem with the coloring scheme is that it does not identify subclasses that 
comprise a color’s usage. For example, areas shaded green depict vegetation of military 
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significance, but the chart may not describe or differentiate between kinds of vegetation, 
such as triple canopy jungle or mangrove swamp (Gorski, 1999). Therefore, the 
inadequate terrain description contained in the map could lead to poor choices in the 
planning process. 

3. Strengths/Weaknesses 

Military maps are excellent planning tools because they offer a common medium 
for forces executing widely disparate functions. They provide a baseline for information 
in a standardized format regarding an area of interest. All combat forces receive map 
reading and navigation training as part of their initial military education, such as boot 
camp or officer candidate school. 

The weaknesses associated with this format include accuracy, information 
latency, and coverage. As a scaled approximation of the Earth’s surface, it cannot 
account for every possible feature or it would become too cluttered to be of any utility. 
Therefore, the planner must also ask the question, “What information has been left off the 
map?” In addition, inaccuracies associated with scaling and data extrapolation make the 
topographical map too imprecise for planning precision-guided munitions employment 
(Tenet, 1999; Pickering, 1999). Information latency occurs when differences between the 
maps representation and reality are caused by changes over time. Typically, the 
information contained in a map that has not been regularly updated with pen and ink 
changes is at least three to five years old (Gervais, 1999). When US forces commenced 
Operation Desert Storm, the most current maps in stock were produced during World 
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War II. Consequently, one of the first support actions that occurs during any crisis is 
updating the maps (Bernard and Trainor, 1995). Commercial development and natural 
weather phenomena also change the terrain, making the data displayed on the map 
inaccurate. Littoral areas that amphibious forces might operate in are especially prone 
these effects. This information difference between the maps and the actual environment 
may lead to a poorer mission plan. Finally, worldwide map coverage does not exist at all 
scales (FM 21-26, 1993). As resources to update maps are limited, emphasis is placed on 
updating and creating maps where US forces currently deploy or will execute most 
operations. 

Imagery formats become an important source in regions where inaccurate, 
incomplete map coverage exists. However, before discussing any individual imagery’ 
format, some of the issues common to all formats are reviewed. 

B. IMAGE FORMAT COMMONALTIES 

All of the images that the analyst interprets are captured by remote sensing 
systems, which record the intensity of electromagnetic radiation that objects reflect, emit, 
or scatter in a particular wavelength band (Sabins, 1997). As the sensor records data, it 
captures spatial characteristics about the Earth’s surface (Aerospace Corp., 1998). Figure 
5 displays the portion of the electromagnetic spectrum exploited by remote sensing. The 
multicolored section of the figure is a graph of atmospheric transmittance as a function of 
wavelength. The dark shaded areas are absorption bands where the various elements and 
compounds found in the Earth’s atmosphere reduce the amount of available energy for 
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the sensor to collect. The light areas are places where electromagnetic radiation passes 
through the atmosphere with little or no absorption. (Short, 1999). The upper portion of 
the figure characterizes the portions of the spectrum. 



V 




Figure 5. Electromagnetic Spectrum Utilized in Remote Sensing (From Short, 1999) 



In each spectral range, the reflectance, or albedo, helps distinguish various objects 
from each other. The relative reflectance of two objects may vary with wavelength. 
Sample signatures for natural and man made materials are provided in Figure 6. For any 
given sensor, the object’s spectral signature partially determines its appearance relative to 
other objects in the scene: The analyst’s ability to distinguish and measure objects in an 
image is also dependent on the object’s size, sensor quality, and contrast level between 
the object and the background (MUG, 1995). 
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Figure 6. Sample Spectral Signatures (Short, 1999) 

1. Image Properties 

The primary properties that all image formats possess are scale, brightness, tone, 
contrast, and resolution (Sabins, 1997). Scale and contrast are ratios. Scale is the ratio of 
image distances to their physical counterparts, and contrast is the ratio between the 
highest and lowest brightness values in an image. Brightness and tone describe 
characteristics of the returned electromagnetic energy. Brightness is a measure of the 
reflected energy intensity at a particular wavelength. Tone quantifies distinguishable 
variations in electromagnetic energy (Sabins, 1997). Resolution is a metric for sensor 
data quality with respect to a given dimension (MUG, 1995). 

Each of the properties described above has a different meaning depending on 
sensor type, and analyst’s expert knowledge of these information assists in interpreting 
the scene. For example, the analyst will interpret brightness differences three different 
ways depending on whether a panchromatic, thermal IR, or radar sensor produced the 
source image. In a panchromatic image, brightness captures the ability of different 
objects to reflect solar energy; in general, better energy reflectors are poorer energy 
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absorbers. As a result, brighter objects in panchromatic images are cooler than darker 
ones. However, in the thermal IR warmer objects provide a stronger return and therefore 
appear brighter in a scene. Brightness in a radar image is a function of the object’s 
geometric shape with relation to the sensor (Sabins, 1997). Personnel with little or no 
interpretation experience might assume that any image they receive is a photograph. This 
mistake could lead to confusion in interpretation of the image and subsequently result in a 
flawed mission plan. 

2. Metrics of Quality 

Resolution is a measure of the quality of the information extracted from a sensor 
with respect to either the spatial, radiometric, or spectral dimensions (MUG, 1995). 

u. GSD 

Spatial resolution or Ground Separation Distance (GSD) is the minimum 
required distance between two objects for those objects to appear distinct and separate. 
GSD is expressed in units of distance and can be approximated by Formula [1] (Sabins, 
1997). Therefore, an image with better GSD allows the analyst to see objects that are 
closer together. 

GSD « (7,/D) H, where: [1] 

X = electromagnetic energy wavelength captured 
D = optic diameter 

H = slant distance from the object to the sensor 
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b. Radiometric 

Radiometric resolution is a measure of sensors’ sensitivity to differences 
in radiance and is expressed as a number of bits (MUG, 1995). For n bits, 2 n independent 
states can represent the energy at a point. Using more bits to represent a single pixel 
increases the number of states, increasing accuracy of the display and lowering the 
quantization error associated with digitally encoding data (Tomasi, 1998). It also 
increases the transmission requirements for the data, because all the bits are required to 
display that pixel. Therefore, better radiometric resolution provides the analyst a more 
accurate image to interpret. 



c. Spectral 

Spectral resolution is a description of what energies are being collected 
along the electromagnetic spectrum and how precisely they are measured. Figure 7 
displays spectral resolution for multispectral and hyperspectral imagery. More 
wavelength bands allow the analyst to process via non-literal means. 
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Figure 7. Spectral Resolution Comparison for Spectral Imaging Sensors (From MUG, 

1995) 
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d. Interpretability 

In the past, spatial resolution was the most often quoted metric for 
information quality, but it has been replaced by interpretability for determining the 
analyst’s ability to correctly answer questions regarding a scene (Aerospace Corp, 1998). 
Figure 8 displays four images in two different formats that have the same GSD. A 
panchromatic imager produced the images in the upper row; a multispectral imager 
produced the images in the lower row. Images from systems in the left column exhibit 
good contrast and low noise, while images from the systems in the right column exhibit 
high noise and low contrast. NIIRS and MSIIRS were developed to give analysts a 
method to convey the interpretability of an image to others (IRARS, 1995). 

(1) NIIRS. The National Imagery Interpretability Rating Scale 
(NIIRS) is a standard adopted by multiple communities including, remote sensing and 
intelligence. Initially developed in the 1970s, NIIRS is a numerical scale for rating image 
quality based on independent textual criteria. The scale initially focused on military- 
related criteria and was later expanded to include civil applications. The basis for the 
rating criteria is image interpretation tasks that the analyst performs routinely. NIIRS 
scale values range from 0 to 9. The higher the numerical value assigned to an image, the 
more information contained in the image. The top left image in Figure 8 would be 
between a NIIRS 5 and 6. The image is closer to NIIRS 6 because the aircraft on the 
tarmac are easily identifiable (NIIRS 5) but car types such as sedans or station wagons 
(NIIRS 6) cannot be determined. The top right image would be rated as NIIRS level 4 



33 




Figure 8. Images with Identical GSD and Varying Image Quality (IRARS, 1995) 

(2) MSIIRS. The Multispectral Imagery Interpretability 
Rating Scale (MSIIRS) was modeled after NIIRS in 1993 to provide a common metric 



because the wing type differences between the small fighter aircraft on the tarmac are 
identifiable and less information can be extracted from the rest of the image (IRARS, 
1996). 
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system for manual exploitation of multispectral imagery in the wavelength range of 0.4 to 
2.5 pm. Unlike NIIRS, MSIIRS is not a community-sanctioned standard. MSIIRS 
values have a range of 0 to 7, based on a generic scale with additional criteria for a wide 
variety of specialized interpretation tasks such as urban, military, or natural. The scale is 
less concrete than NIIRS because the image analysts are free to exploit the image using 
any band combination they choose. The lower left image in Figure 8 could be assigned a 
MSIIRS level 5 because individual cars are identifiable in the parking lot located just 
above the slightly wooded area (IRARS, 1995). While NIIRS and MSIIRS are similar 
concepts, they have completely separate criteria that are not meant as a cross-reference 
between images of different formats (IRARS, 1995; IRARS, 1996). 

C. TRADITIONAL IMAGERY 

The traditional imagery available to the imagery analyst and mission planner 
comes from airborne and space platforms, in either soft or hard copy. If in hard copy, the 
material may be a paper product, or a transparency that can be viewed on a light table. 
The trend today is towards soft-copy, which can be exploited on a computer, and 
manipulated using a variety of image processing tools. Traditional imagery formats are 
panchromatic, infrared, radar and single band MSI. 

1. Panchromatic Imagery 

Panchromatic sensors record reflected solar energy as a single spectral band in the 
visible region, approximately 0.4 to 0.7 pm, of the electromagnetic spectrum (Sabins, 
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1997). While understanding how objects reflect or absorb solar energy in this region is 
important, the importance is reduced for interpreting panchromatic imagery because 
much of the general knowledge the interpreter uses in this format is based on human 
experience. The brightest areas in an image represent objects that reflect or scatter 
incident energy best within a scene. Conversely, objects that absorb energy best within a 
scene are the darkest areas of a picture. If a panchromatic scene contained the objects 
whose spectra are described in Figure 6, the red sandpit would be the brightest object in 
the gray scale display, followed by pinewoods and grasslands. The darkest object in the 
scene would be the silty water (Short, 1999). The gray scale differences can then be 
grouped together into different ways that allow the analyst to use other elements of 
recognition to interpret the scene. 

If the analyst has two images of the same area that differ only in perspective, he 
may choose to interpret the image using stereoscopy. The technique takes advantage of 
the human visual system to produce a three-dimensional image in the analyst’s brain by 
using two two-dimensional images and a stereoscope (Sabins, 1997). Without 
stereoscopy, the analyst can obtain only limited information about terrain relief. In this 
case, marginal terrain relief can be obtained by analyzing how the sun strikes the earth in 
the image or from object layover. 

Object layover make the tops of taller objects within a scene appear to lean away 
from the optical center of the image. The effect is created because the photons earn ing 
information about the top of an object arrive at the sensor from a different angle than the 
photons carrying information about the bottom of the structure. Since the sensor can only 
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generate a two-dimensional image, the photons from the object are transposed onto the 
image plane away from the optical center of the image. Figure 9 displays an aerial 
photograph with layover and a pictorial description of the effect. Look angle, object 
height, and relative displacement from the optical center of the image will all affect the 
level of distortion created. A trained analyst uses layover as a tool to obtain relief data 
using simple geometry (Sabins, 1 997). 




Figure 9. Example of Layover in Aerial Photography and Explanation (Sabins, 1997) 



The strongest advantage of the panchromatic format is that the analyst can easily 
describe the final assessment in terms the mission planner understands and trusts. 
Panchromatic images also provide more quantitative and qualitative intelligence about an 
area than a map does (FM 21-26, 1993). However, daylight and fair weather are essential 
for successful imaging (Sabins, 1997). Obscurants such as water vapor or smoke render 
panchromatic sensors useless because the sensor captures the reflected solar energy of the 
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obscurant, not the target of interest. In addition, raw images will not contain precise 
location data when compared to a map (FM 21-26, 1993). 

2. Infrared Imagery 

The infrared portion of the electromagnetic spectrum can be divided into two 
major sections. Reflected infrared includes the near infrared (0.7 to 1.2 pm) and short 
wave infrared (1.2 to 3.0 pm) regions of the electromagnetic spectrum. Thermal or Long 
Wave Infrared is located from 3 to 15 pm the electromagnetic spectrum. The wavelength 
band from 5-8 pm is an atmospheric absorption band of electromagnetic energy (Sabins, 
1997). 



a. Reflected IR 

Solar energy dominates the reflected IR region. An object’s appearance in 
gray scale is based on its reflectance values in this region, which is the same phenomenon 
as panchromatic. However, the amount of spectral reflectance for different objects can 
vary from the visible region. One such example is vegetation. The solar reflectance for 
certain vegetation types dramatically increases at approximately 0.8 pm. Using this “IR 
ledge” is one method of finding camouflaged objects. True vegetation in this region is 
very bright, while cut vegetation and camouflage material appear dark (MUG, 1995). If a 
scene containing the objects of Figure 6 were imaged in the reflected IR, grasslands 
would be the brightest objects in the scene, followed by pinewoods and the red sandpit, 
with silty water still being the darkest object (Short, 1999). The scene will not appear the 
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same as a similar image in the visible region. This effect can be confusing to untrained 
personnel and may lead to a poorer mission plan. 

b. Thermal IR 

Observations at longer wavelengths are controlled by the thermal emission 
from the targets, and the scene dynamics differ from those described above for the 
reflective IR domain. Figure 10 is a graph of Plank black body radiation formula for two 
objects (Gettys, 1989). The dashed line depicts ideal blackbody solar energy at the 
Earth’s surface. The solid line depicts an object with an approximate temperature of 
300°K. Energy from solar emissions dominates the graph up to approximately 3-5 pm, 
after which object emission dominates. Hence, FLIR and other observations in the 
MWIR and LWIR are largely independent of solar illumination. Another advantage 
thermal IR sensors provide over panchromatic is that airborne particles such as smoke 
and dust have a reduced effect as an obscurant because the particles themselves are 
smaller than the energy wavelengths of interest (Sabins, 1997; Aerospace Corp., 1998). 

Image characteristics will often appear similar to panchromatic imagery’ 
because both formats record information as gray-scale variations. The brightest tones on 
thermal IR images represent the objects with the most radiant energy, while darker tones 
correspond to objects with the least radiant energy. This explanation is still incomplete, 
because two objects such as wood and metal that have the same absolute temperature 
would nominally radiate the same amount of energy, and therefore they would be 
indistinguishable to the analyst. Fortunately, just as the reflective character of targets 
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varies from material to material, so does the “emissivity,” or the relative ability of an 
object to emit radiation, in comparison to an ideal “black body.” These variations are 
relatively subtle (a few percent, typically) but sufficient to enhance the utility of IR 
sensors in distinguishing targets that are otherwise not distinguishable. The object’s 
other thermal characteristics such as inertia, conductivity, and capacity determines an 
object’s ability to absorb and retain heat. In turn, these attributes as well as atmospheric 
effects impact how an object appears relative to its environment in a given image (Short, 
1999). 




Figure 10. Planck’s Blackbody Radiation Curves for the Sun and the Earth Measured at 

the Earth’s Surface (After Gettys, 1989) 



40 



Several phenomena on thermal IR images can be confusing to the 
untrained user. Figure 1 1 displays diurnal temperature variations for various objects that 
could comprise a scene. In the early morning hours and in late evening, the brightest 
objects in the scene are vegetation, followed by standing water, rocks/soil, damp terrain, 
and metallic materials. However, between local sunrise and local sunset rocks and soil 
become the most radiant objects in the scene so they would appear the brightest (Sabins, 
1997). In addition, wind patterns that cool selected objects can produce streaks on the 
image that could be interpreted as significant tonal variations. Therefore, without a basic 
understanding of these effects, image misinterpretation is possible. 




Figure 11. Diurnal Temperature Variations for Selected Objects (Sabins, 1997) 

System resolution is another challenge the analyst faces when interpreting 
thermal IR images. Formula [1] shows that the GSD is proportional to the collected 
wavelength X. Since thermal IR wavelengths are an order of magnitude longer than 
visible wavelengths, the GSD between comparably sized objects must increase by the 
same amount in order for those objects to appear distinct, for a given optical system. 
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Therefore, an analyst who relies heavily on GSD can be hampered in his or her efforts to 
interpret thermal IR imagery. This effect is illustrated using LANDSAT TM imagery in 
Figure 13. Bands 1 and 2 are both images in the visible portion of the spectrum while 
Band 6 images the thermal IR. The band 6 image is clearly less sharp than Bands 1 and 2 
(Short, 1999). 

The strengths of using thermal infrared images include the fact that the 
medium captures information outside the range of the human eye. Sensors do not require 
solar illumination for imaging and weather has a reduced impact on this format when 
compared to panchromatic. The major disadvantage associated with this format is the 
confusion an untrained user might encounter. 

3. Radar Imagery 

Radar imaging systems create an image by transmitting electromagnetic energy 
from selected wavelengths from 0.1 to 100 cm, then capturing the reflected 
electromagnetic energy. The long wavelengths would nominally lead to very poor 
resolution given the applicability of equation [1]. Increased spatial resolution is attained 
by using a very large antenna, which can be produced a virtual method such as synthetic 
aperture radar (Sabins, 1 997). 

Radar sensors provide their own illumination. Therefore, they are not dependent 
on solar reflection or thermal emission for successful imaging. In addition, atmospheric 
effects such as absorption in this portion of the electromagnetic spectrum are 
substantially reduced. 
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Pixel brightness in a radar image indicates spots where the transmitted energy was 
returned to the sensor. Objects with steep slopes towards the antenna produce strong 
returns, thus bright pixels. Comer reflectors such as buildings produce the strongest 
returns. Objects with steep slopes facing away from the antenna produce very low 
returns or dark pixels, because no energy reaches the area. Objects that scatter energy in 
all directions such as vegetation produce intermediate returns. Smooth surfaces such as 
pavement or water reflect energy away from the antenna thus producing very small 
returns (Sabins, 1997). Figure 12 is a radar image of the Pentagon that shows these 
effects. The exterior shape of the Pentagon is very strong, while the roads at the bottom 
of the image are very dark. The vegetation in the courtyard at the center of the Pentagon 
produces the intermediate tones described. 

Smoothness is relative to the imaging system wavelength. An object such as 
pavement appears rough in the visible region because the pavement scatters, rather than 
reflects, most of the incident energy. However, the radar’s wavelengths are thousands of 
times greater than those in the visible region, so the pavement reflects most of the energy 
away. A surface is considered smooth if it meets the criterion shown in Formula [2] 
(Sabins, 1997). 

h < X (8 sin y)' 1 , where: [2] 

h = vertical relief 
X = wavelength 

y = depression angle (angular measure of how far down the sensor 
is pointed from the horizontal) 
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Phenomena such as object layover, relative motion, and topographic inversion also 
present challenges to interpreting radar images. Radar images also experience object 
layover, but it is reversed from the perspective found in panchromatic. When an object 
with large vertical relief is imaged, the reflected energy from the top of the object arrives at 
the sensor earlier than energy from the bottom of the object; therefore, the object appears to 
lean toward the sensor. When the object sensed is in motion, distortion or smearing can be 
produced because there are multiple returns in different places for the same object (Sabins, 
1997). Topographic inversion is produced when the brain unsuccessfully guesses the 
relative location of an illumination source when attempting to determine terrain relief. An 
unsuccessful guess causes ridges to look like valleys and vice versa (Sabins, 1997). 




Figure 12. Radar Image of the Pentagon (FAS, 1997) 
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The major advantage of radar images is that they are completely independent of 
both lighting conditions and weather. The major disadvantage associated with radar is the 
same as Thermal IR; the image format requires increased training to properly interpret. 



4. Single Band Multispectral Imagery (MSI) 

Multispectral imagery is on the boundary of current operational tools and the 
specialized analysis domain of the expert, such as the mapmakers at NIMA. Still, 
imagery analysts can utilize multi-spectral imagery, such as that obtained from the 
LANDSAT satellites. Its distinction from traditional military systems is that 
multispectral sensors collect the energy intensity simultaneously from two or more 
contiguous regions of the electromagnetic spectrum, recording the intensity values 
separately (MUG, 1995). MSI sensors probe several transmission windows in the .4 to 
13 pm region of the electromagnetic spectrum. The collected energy is some 
combination of solar energy and thermal radiation that varies as a function of wavelength 
(Aerospace Corporation, 1998). For example, the LANDSAT Thematic Mapping (TM) 
Satellite has seven spectral bands (values in pm): 1(0.45-0.52), 2(0.52-0.6), 3(0.63-0.69), 
4(0.76-0.9), 5(1.55-1.75), 6(10.4-12.5), and 7(2.08-2.35) (Sabins, 1997). Interpreting one 
band of LANSAT imagery independently of the other bands allows the image to be 
displayed as a gray scale image. 
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Figure 13. LANDSAT TM Imagery of Morro Bay, CA (Short, 1999) 



The environmental characteristics the sensor responds to are a function of the 
band interpreted, which means the same scene will appear slightly different in each band. 
Figure 13 displays this effect for Morro Bay, California. For example, LANDSAT TM 
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bands 1 and 2 are both in the visible region, but band 1 is more sensitive to natural blue 
light, while band 2 is more sensitive to natural green light. Neither band is sensitive to 
red or infrared light. Since the analyst understands this, he or she might use this 
technique to locate or describe objects that have high contrast in one of these bands. 
LANDS AT bands 4, 5, and 7 responds to the reflected IR energy and band 6 respond to 
thermal IR differences in a scene. 

5. Summary 

To summarize, when an image that represents a portion of the Earth surface 
contains information that comes from a single spectral region, it may be interpreted 
literally in gray scale. Each sensor encodes the information into gray scale differently. 
Therefore, the analyst must understand the method the sensor uses to capture and record 
the energy as well as certain physical effects to prevent misinterpretation. 

D. SPECTRAL IMAGERY 

In certain cases, a single gray scale image does not give the analyst enough 
information about the area of interest to meet all the planner’s requirements. While a 
multispectral sensor may physically capture the photons only once, it creates as many 
images as there are bands. By performing additional processing on the scene, it is 
possible to combine the effects of multiple bands into a single image that generate 
additional visual cues for the analyst to interpret. If the additional processing is not 
performed, it is likely that the resulting scene will provide no additional information 
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about the scene than any other original images independently. Each of these formats 
(MSI, HSI, and sensor fusion) gives the interpreter the ability to analyze the image using 
literal and non-literal means. 

1. Multispectral Images (MSI) 

Multispectral image interpretation allows the user to combine the several spectral 
bands to create image products that differ significantly from the single band gray scale 
images described previously. Coupling the information in the bands with object spectral 
reflectance characteristics can result in higher information extraction when compared to a 
panchromatic image (Sabins, 1997). The methods for interpreting MSI include true and 
false color composites, classification, and principal components analysis. 

a. True Color Composites 

Joint Intelligence Centers have the capability to produce products such as 
true color composites. While the production method falls into the realm of spectral 
analysis because the analyst combines information from several spectral bands to produce 
the image, the end product is an image that can be interpreted literally without additional 
training. Color, shape, and size constancies are easy to establish, because the objects in 
the scene appear as they do in human experience. This sort of product can be an 
excellent planning tool for the military if the season in which the sensor captures the data 
coincides with the season the operation in which occurs. 
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A natural extension of the single band multispectral image interpretation 
described earlier is to integrate color as a method of conveying information. The most 
common images produced using MSI data are three-band color composites, which assign 
each a band different color gun (MUG, 1995). Each pixel usually has three digital 
numbers assigned: red, green, and blue. After each color gun is fired, the colors mix in 
an additive manner for each pixel on the analyst’s computer monitor. 

A true color image matches the spectral data for red, green, and blue 
information to the red, green, and blue color guns. Figure 14.a displays a LANDSAT 
TM true color image of Morro Bay, CA (Short, 1999). Most objects in the image appear 
as they would by direct human observation, such as from an airplane window. The water 
around the coast is blue, the golf course on the east coast of the bay just south of the inlet 
is green, and the urban area is characterized by the light gray grid of streets, as are the 
two highways coming and out of the town (Short, 1 999). 

However, certain objects in Figure 14. a still do not appear the way they 
would through direct observation. For example, the Pacific Ocean to the west of Morro 
Bay is still too dark because water vapor in the atmosphere absorbs energy from this area 
before it reaches the sensor (Short, 1999). The most unrealistic color in the scene is the 
hill slope. It should be a golden brown, but is displayed with a purple tone. The reason 
for this color distortion is grounded in physics. Each band of energy in the image reacts 
differently to refraction, reflection, and absorption in the Earth’s atmosphere. 
Subsequently, greater or lesser amounts of that energy arrive at the sensor than the value 
required to “mix” the proper color. In Figure 14.a, more red light arrives at the sensor 
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than blue and even less green, which creates the color mixture displayed (Short, 1999). 
Table 3 provides a textual listing of how other features are displayed in true color images. 





a. True Color Bands 1, 2, 3 (BGR) 



b. NIR False Color Bands 2, 3, 4 (BGR) 





c. False Color TM Bands 5, 7, 6 (BGR) d. False Color TM Bands 4, 7, 1 (BGR) 
Figure 14. True and False Color Composite Images of Morro Bay, CA (Short, 1999) 
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b. False Color Composites 

Whenever the analyst selects a color scheme that does not equate the red, 
green, and blue (RGB) color guns to RGB information in the image, a false color image 
is generated. Any color image that displays spectral information that the human eye 
cannot perceive as a color is false color. Near IR (NIR) and SWIR are the most common 
formats for MSI pictures. Table 3 describes the color gun assignments for NIR and 
SWIR false color composites as well as a textual description of how objects are colored 
in these images (MUG, 1995). 





True Color 


Near IR 


Short Wave IR 


Blue Gun displays 


visible blue 


visible green 


visible red 


Green Gun Displays 


visible green 


visible red 


near IR 


Red Gun displays 


visible red 


near IR 


short wave IR 


Trees and Bushes 


olive green 


red 


shades of green depending on 
band color gun combinations 


Crops 


medium to 

light green 


pink to red 


shades of green depending on 
band color gun combinations 


Wetland Vegetation 


dark green to 
black 


dark red 


shades of green depending on 
band color gun combinations 


Water 


shades of blue 
and green 


shades of blue 


black 


Urban Areas 


white to light 
blue 


blue to gray 


lavender 


Bare Soil 


white to light 
gray 


blue to gray 


magenta, lavender or pale pink 



Table 3. General Appearance of Features in Various Composite Images (MUG, 1995) 
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Figure 14.b displays a false color composite of Morro Bay, CA. 
LANDSAT TM Band 2, which is natural green light, was assigned to the blue color gun. 
Band 3, which is natural red light, was assigned to the green color gun. Finally Band 4, 
which is reflected IR, was assigned to the red color gun. This corresponds to what one 
would see with traditional “IR” film. The previous discussion of reflected IR described 
how healthy vegetation appears very bright in the reflected IR. This correlates to the 
bright red colored areas in the image. Based on the level of red tint in different sections 
of the scene, the analyst can coarsely classify vegetation types (Short, 1999). 

In addition, the nearshore areas of the Pacific Ocean in the scene are 
lighter because the sediment in the water adds reflectance to the scene. In planning an 
amphibious mission, the planner would want to avoid areas where he or she could see 
blue very far away from shore because it could be an indication of shallow slope which 
traditional displacement landing craft may run aground on. This shallow slope problem 
was one of the concerns that planners during Operation Desert Storm faced in preparing 
the amphibious assault of Kuwait (Gordon and Trainor, 1995). 

Figures 14.c and 14.d are two other examples of false color images. 
Figure 14.c is used to convey information about thermal differences based on band 
selection. Just as in other relative temperature plots, shades of blue represent the coolest 
areas of the image, while the red shades represent the warmest colors in the image (Short, 
1999). Specific temperatures are not assigned to the color hues, so this information could 
be misleading to a planner if the temperature difference were only 10°C. Figure 14.d 
shows each of the major features of the image in a primary color. Vegetated areas are 
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displayed in bright blue; bright reds classify the surf, towns, and other hydrographic 
characteristics; greens are uncultivated areas (Short. 1999). 

Figures 14.c and 14.d illustrate a major drawback in false color image use 
for military' planning; they are disturbing to the untrained user because they violate the 
concepts of color constancy. People expect water to be blue and grass to be green 
regardless of illumination conditions. When objects such as these appear in a different 
color without explanation, the viewer can become confused. If the image analyst cannot 
explain what the color scheme means in a concise fashion that the planner can understand 
and apply, then the planner will probably be thrown off by the fact that the colors are 
wrong and be less likely to use that information in the planning process. The US 
Department of Defense (DOD) appears to have minimized this problem in MSI by using 
only certain false color composite image types. With the number of types minimized, the 
analyst and the planner can use keys similar to Table 3 to interpret how different features 
appear in different color groupings (MUG, 1995). 

c. Digital Image Processing 

There are several digital processing techniques available for the analyst’s 
use in order to extract information from a scene. Principle Components Analysis and 
Classification are just two of the digital image processing techniques that take advantage 
of the fact that there are several DNs for each pixel, each representing different spectral 
information. 
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If the individual images in Figure 13 are quickly reviewed, they all appear 
very similar except band 6. Principal Component Analysis (PCA) is one processing 
technique that uses all available spectral bands to create the same number of new images. 
However, the first few principal component (PC) images contain most of the variability 
between bands and most of the remaining images are noise. If materials within a scene 
have significantly different spectra, this may aid in target discrimination. This effect can 
be seen in the seven PC images of Morro Bay in Figure 15. PC 1 is generally a weighted 
average of all the spectral bands that looks like a black and white photograph. PC 7 and 
possibly PC 6 can be considered noisy. Additional PC images highlight different features 
in the scene that the analyst can literally interpret, provided the analyst has the associated 
knowledge to accompany the image (Richards, 1999). 

The PC images may also be combined into a color composite. PC color 
composites utilize more of the available color because each PC image highlights 
uncorrelated differences, while the standard color composite does not (Richards, 1999). 
Figure 1 5 displays a PC color composite by assigning the information from PC images 1 , 
2, and 3 to the blue, green, and red color guns, respectively. 

An advantage of PCA is that only the first few images require 
interpretation. Another advantage is that sensor noise that is inherent in all bands is 
reduced. Therefore, the multi-band data set is effectively reduced to a single, lower-noise, 
gray-scale image that is suitable for transport over low bandwidth communications 
circuits (Short, 1999; Sabins, 1999). A disadvantage is that if the analyst or planner does 
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not understand how this processed image differs from the original, then the advantage of 
PCA is lost. 





PC 1 




PC 5 
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PC Color Composite- 
images 1, 2, 3 (RGB) 



Figure 15. PC Component Images of Morro Bay, CA (Short, 1999) 



Classification groups pixels in a scene based on known spectral class 
criteria. Selecting n bands from the available data sets creates an n-dimensional 





classification space. Each band becomes an orthogonal axis to the others. The pixels in 
the scene are mapped into the space based on their DNs for each band and can then be 
grouped by comparing them to known material spectra. Assigning each class a color and 
re-mapping the pixels geo-spatially creates a new color image (Short, 1999). Some 
pixels will remain unclassified because they do not meet the criteria of any class 
particularly well. In this case, they may be assigned to a class by way of statistical 
analyses. If the classification is supervised, an analyst can control assignment; otherwise, 
the computer can perform this automatically (Short, 1999; Sabins, 1997). 

d. Summary 

The major advantage of MSI is that the analyst can use the spectral 
characteristics of objects to aid in their classification. By doing so, he or she provides 
information about the area of interest that might not have been previously available in 
any other format. Using color encoding allows the analyst to coarsely classify soil 
vegetation types and water characteristics. Like infrared and radar formats, MSI provides 
information outside the range of the human eye. 

On the other hand, MSI is a complex medium requiring expert knowledge 
in all of the electromagnetic regions described so far, as well as an understanding of how 
to properly manipulate the data. Digital storage and processing requirements increase 
with MSI, because each uncompressed scene contains one image per band. Provided the 
data is collocated with planners and manipulation tools exist, this is not a problem. 
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However, the limited data rates that shipboard planners have access to make it difficult to 
transfer a complete scene electronically. 

2. Hyperspectral Images (HSI) 

The distinguishing feature between MSI and HSI is the number of bands. While 
MSI sensors utilize up to seven spectral bands, SI sensors generate hundreds of narrower 
spectral bands in a spectral region (Aerospace Corp., 1998). Figure 7 pictorially displays 
the spectral difference between MSI and HSI for a typical portion of the electromagnetic 
spectrum (MUG, 1995). 

The data returned by hyperspectral sensors are usually referred to as a 
hyperspectral “data cube,” such as the one displayed in Figure 15. Two of the three cube 
dimensions are spatial coordinates (e.g. x, y) and the third dimension is wavelength X 
(Aerospace Corp., 1998). Therefore, a single (x, y, X) value describes the spectral 
response for position (x, y) at wavelength X. Fixing the spatial coordinates in a scene on 
a single pixel, then plotting the extracted reflectance data as a function of wavelength 
produces a pixel vector. This trace is similar to the spectral characteristic curves in 
Figure 6 (Stefanou, 1997). 




Figure 16. Sample Hyperspectral Data Cube (From Short, 1999) 
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The data cube may also be manipulated to produce monochromatic or color 
composite images, which can be interpreted literally. In addition to the literal techniques, 
there are also non-literal techniques that can be used to extract information from the data 
cube. 

a. Non-literal Processing Techniques 

The higher spectral resolution provides the opportunity to detect subtle 
spectral differences that are too narrow to be differentiated using MSI (MUG, 1995). 
These differences in reflection curves are not resolvable using the literal analysis 
techniques using panchromatic imagery (Short, 1999). Principal Components Analysis— 
which was discussed in the multispectral section— Spectral Angle Mapping, and Spectral 
Matched Filter are three basic non-literal interpretation techniques used on hyperspectral 
data cubes. Stefanou (1997) provides a more in-depth description of additional HSI 
processing techniques. 

SAM and SMF were performed on hyperspectral data of a cove at Eglin 
AFB, FL. Figure 17 is a true-color composite of the cove, which is provided to orient the 
reader to the region. The target spectra selected from the data cube were (color coding in 
parenthesis): vegetation (green), shadows areas in vegetation (red), deep water (blue), 
shallow water (yellow), sand (brown), and asphalt (maroon). The spectral library of 
Figure 1 8, which was used to analyze the rest of the dataset, were developed by selecting 
pixels that were comprised of that target material from Figure 17. Figure 19 shows the 
pixels selected to produced the library overlaid onto the second principal component 
image (PC 2) for the data set. 
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Reflectance 




Figure 17. True Color Composite of Eglin AFB (Thanks to Chris Simi, Night Vision Lab) 




Figure 18. Mean Spectra for Regions of Interest 
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Figure 19. Regions of Interest for Classification Overlaid on PC 2 

(1) Spectral Angle Mapping (SAM). SAM is a deterministic 
matching algorithm that compares pixel spectra to selected target spectra by comparing 
the angle between pixels when both are mapped into N-dimensional space (Richards, 
1999). When the angle value is within a user-defined range (expressed in degrees or 
radians), the pixel is identified with the material that has those characteristic spectra. A 
composite image can then be created where the different elements in a scene are color 
coded, providing a material or target map. Figure 20 is an target map produced by 
performing SAM on the data set. The selected angle value was 0.1 radians. Increasing 
the angle value could have reduced the number of unclassified pixels. However, 
increasing the angle value also increases the chance of mislabeling a pixel. 
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Figure 20. Target Map of Eglin AFB Produced by SAM 



(2) Spectral Matched Filter (SMF). SMF is a technique that 
is representative of a class of statistical methods. It is derived from signal processing 
theory (Richards, 1999). SMF produces an output that is proportional to an estimated 
contribution of a target material to a given pixel (Collins, 1 996). It is important to note 
that both SMF and SAM require a comprehensive spectral library for the analyst to 
conduct a comparison of the portions of the scene (Richards, 1999). Figure 21 illustrates 
the image produced from SMF. Brighter pixels indicate a better likelihood that that pixel 
is comprised of the target spectra. These individual SMF images can also be combined 
into color composites. Figure 22 is a SMF color composite produced by assigning the 
deep water image to the color blue, and assigning the shallow water to the colors red and 
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green. In addition, the SAM data for vegetation, sand, shadow areas around vegetation, 
and asphalt were over overlaid onto the image. 




Figure 21. SMF- Vegetation Class Probability 
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Deep Water 
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Shadow areas near vegetation 

Vegetation 

Sand 

Asphalt 

Figure 22. SMF Color Composite with Selected SAM Data Overlaid 
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b. Interpretation Challenges 

A new set of problems arises when the analyst literally interprets a three- 
band HSI composite or the products of one of the exploitation techniques described 
above. The usage of false color images as with MSI is exacerbated because there are no 
“standard” composite images in HSI (MUG, 1995). In addition, when the analyst selects 
three portions of the spectrum for analysis, he or she in effect omits all but those three 
portions in the interpretation, thus negating the benefits of higher spectral resolution. 

Analysis of spectral imagery is a discipline that is still under development, 
and there are a number of problems with the use of such data. Deterministic techniques 
require a complete spectral library of background and target materials. Statistical 
methods can distinguish materials, but may not identify them. Techniques such as SMF 
can extract targets from a background but produce images that give probabilities of 
occurrence that can be difficult to interpret (Stefanou, 1997; Short, 1999). 

c. Incorporating HSI into Existing Capabilities 

While HSI technologies have found applications in the commercial sector 
(Neuenschwander, 1998), such as in the environmental monitoring function, they are still 
not an operational capability within the US Department of Defense. However, HSI is 
considered an advanced technology concept, covered under the Common Spectral 
MASINT Exploitation Concept (COMSEC). The COMSEC’s purpose, as stated in the 
Joint Warfighting Science and Technology Plan (1999) is quoted below: 
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“...to demonstrate COSMEC, end to end, to an operational user, showing the tactical 
utility of MASINT spectral analysis to the warfighter. This ACTD will establish 
COSMEC’s ability to support the Joint Vision 2010 mission areas of Information 
Superiority and Combat Identification, as well as supporting specific operational 
requirements. It will provide processing and exploitation capability to analysts in 
preparation for government and commercial multi/hyperspectral collection platforms. 
COSMEC supports both tactical and strategic intelligence using state-of-the-art MASINT 
processing and exploitation algorithms. These algorithms will enhance the U.S. spectral 
data exploitation capability. COSMEC has the ability to support a variety of operational 
requirements, including detection and ID of camouflage and vehicles, search and rescue, 
terrain characterization and mapping, beach route preparation, submarine detection, and 
detection of chemical/biological weapons. COSMEC will provide operational units with 
the capability to exploit data from existing and planned spectral sensors like the Land 
Remote Sensing Satellite (LANDSAT), Satellite Pour d’Observation de la Terra (SPOT), 
Senior Year Reconnaissance System Preplanned Product Improvement (SYERS P'T), 
Hyperspectral Digital Imagery Collection Experiment (HYDICE), Spectrally Enhanced 
Broad-band Array Spectrograph ic System (SEBASS), Moving Target Indicator (MTI), 
and Littoral Airborne Sensor Hyperspectral (LASH). The modular design of COSMEC 
will simplify the process of updating the program with new algorithms or sensors, such as 
Warfighter and Naval Earth Map Observer (NEMO). This demonstration of COSMEC to 
an operational unit will establish the ability of multi/hyperspectral analysis to support 
search and rescue or camouflage detection in a combat-oriented mission. COSMEC’s 
user-friendly interface and extensible architecture make it a versatile and useful tool for 
the warfighter.” 



3. Sensor Fusion 

The final method of analysis involves merging data from two or more of the 
formats previously described to produce a separate image for the area of interest. When 
formats are combined properly, the analyst is able to extract more information about the 
area of interest than by using any of the formats separately. A typical format 
combination is MSI and panchromatic. LANDSAT color composites have poor GSD (30 
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m) but are good for classifying vegetation. Similarly, a good panchromatic image from 
SPOT or IRS has better spatial resolution but only for a single wavelength region. By 
digitally combining the panchromatic image with the color composite, the analyst can 
describe the region with panchromatic with its better GSD, as well as classify soil types 
using MSI, using a single image (Sabins, 1997). 

Another example combines MSI and DTED to determine what pieces of land are 
feasible for a particular type of military operation. For example, an airborne drop zone 
must be flat, clear land. The analyst determines suitable locations by combining digital 
terrain elevation data (DTED) and MSI. By using MSI, the analyst finds areas devoid of 
vegetation, then uses DTED to determine the terrain gradient. Areas that meet both 
criteria are identified as suitable drop zones, which can then be included in the planning 
process (Gorski, 1999). 

While sensor fusion shows promise, these are special products produced on 
demand and not used by military planners on a regular basis because of the lengthy 
process necessary to create a single image. Multiple formats for a single area must be 
located before they can be combined. After the individual images are acquired they must 
be co-registered. Common geographic points must be selected to properly align the 
images (Short, 1999). Co-registration in itself can be difficult because of the way objects 
layover in different formats (i.e., objects in radar lie towards the focal point, but objects 
in panchromatic lie away from the focal point) (Sabins, 1997). Completing this process 
in a timely fashion poses difficulties to afloat planners with limited bandwidth and 
computing power. In addition, these products will probably be displayed in false color. 
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As with MSI, the utility of the final product is based on how well the planner understands 
the basis for the analyst’s assessment. 



E. SUMMARY 

This section reviewed the major planning aid formats used to prepare military 
operations. Each format has its absolute strengths and weaknesses, as well as advantages 
and disadvantages relative to other available formats. In most cases involving imagery, 
the planner’s confidence in the analyst’s assessment is largely based on his or her ability 
to understand how the analyst reached a decision. 
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V. 



THE STUDY 



To determine the impact that HSI can have on amphibious warfare planning, the 
impact that existing formats have must be ascertained. The impact that existing formats 
have is evaluated by a study of image analysts. In order provide the analyst quantifiable 
jobs that they can perform on an image that support amphibious mission planning, a task 
analysis was performed on the amphibious planning process. 

A. TASK ANALYSIS 

A task analysis reduces a complex evolution into a set of less complex events that 
may be analyzed independently. The elements identified in the task analysis represent 
the information requirements necessary to plan the mission. This thesis uses the task 
listing method that Zemke (1984) describes to complete the task analysis. 

Task listing captures the duties involved in performing a task by breaking a large 
task, such as planning an amphibious mission, into the primary subtasks involved with 
accomplishing the main task. Subtasks comprise the steps necessary to complete the 
main task regardless of the effort required for completion. In turn, the primary subtasks 
are decomposed into their subtasks in the same manner as the main task was. This 
process of subdivision is then recursively performed on each subtask until the desired 
level of granularity is obtained, which for this thesis was whether the subtask was 
answerable from Imagery Intelligence (IMINT). Subtasks that were not answerable by 
IMINT were omitted from further development following identification. This list of 
subtasks or task elements is provided in Appendix B. 
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B. 



IMAGERY ANALYST STUDY 



The image analyst study was conducted to identify the cognitive processes 
involved with image interpretation in support of amphibious operations. Analysts were 
tasked with completing one of the task elements in Appendix B while being observed. In 
addition, this thesis looked to identify situations where the analyst was uncertain with his 
or her interpretation. 

1. Participants 

The imagery analyst study was conducted at two of the Unified Combatant 
Commanders Intelligence Centers, Atlantic Intelligence Command (AIC) in Norfolk, VA 
and Southern Command Joint Intelligence Center (SOUTHCOMJIC) in Miami, FL. as 
well as the Naval and Marine Corps Intelligence Training Center (NMITC) in Dam Neck, 
VA. The six subjects at AIC were assigned to the Expeditionary' Warfare Products 
Division (DI3). The fourteen subjects studied at USSOUTHCOM were assigned to the 
JIC Imagery Division. The three subjects studied at NMITC were Imagery Analyst ‘C’ 
School instructors. The sixteen male subjects and eight female subjects were military 
image analysts from all of the services in the US DOD or DOD Civilians. The average 
exploitation experience was 7.4 years (range 1-20 years, S.D. 6 years). The average 
length of formal training for the 24 analysts was 1 1 months (range 3-48 months, S.D. 
1 1.6 months). Data regarding age were not collected. Table 4 shows the rank and service 
of the different analysts observed in the study. 
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Rank 


USA 


USN 


USAF 


USMC 


DOD CIV 


E-4 


1 


2 


1 


0 


- 


E-5 


3 


1 


0 


2 


- 


E-6 


2 


5 


1 


1 


- 


E-7 


0 


0 


1 


0 


- 


CIV 


" 


" 


" 


" 


3 


TOTAL 


6 


8 


3 


J 





Table 4. Image Analyst Study Demographics 



2. Apparatus 

Subject interviews were conducted in their work settings while performing their 
normal analytical duties. Analysts exploited high resolution panchromatic, radar, and IR 
littoral scenes derived from National Sensors. MSI from LANDSAT was also exploited. 
The author had no control over the image area, format, or quality presented to the analyst. 
At the JICs, the analysts performed their analysis on recently acquired digital images 
displayed on computer screens. At NMITC, the analysts performed their interpretation 
on hardcopy black and white photographs or acetate negatives using a light table. These 
images were specifically selected for training new analysts. 

3. Procedure 

During the interview each subject was asked how he or she would complete one 
of the subtasks provided in Appendix B. Subjects were free to use any available image 
formats and resource materials such as recognition guides. No time restrictions were 
imposed on the subjects. Interviews lasted approximately 40 minutes. 
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Information regarding the decisions were collected using verbal protocols 
(Carswell, 1999), in which the subject verbally described the steps taken to reach a 
cognitive decision based on what he or she perceived in an image. Verbal protocols were 
used because many of the intermediate steps in image interpretation are second nature to 
the subjects. This technique allows the interviewer to collect data by having the subject 
“talk through” a specific task. Finally, the author asked each subject how confident he or 
she was with his or her result. If they expressed uncertainty, subjects were asked to 
describe how their certainty level was reduced. 

C. ANALYSIS 

The data collected at this point was a list of task elements from the task analysis 
and human factors data describing how trained analysts interpret the most common forms 
of imagery. Qualitative analysis was performed on the data because the small sample 
size and the lack of control the observer had over the situation prevent analysis by 
statistical means. A written description of how an analyst accomplished a particular 
subtask was created. Each subtask description contains information about visual clues, 
identifiers, techniques, formats used, and level of certainty. 

It is hypothesized these data will expose areas where the analyst is less certain 
w'ith the answers provided to particular task elements. It is also hypothesized that data 
from HSI can improve the analysts' certainty in these area and possibly provide valuable 
information that was previously unavailable. The improved environmental knowledge 
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base will increase the mission planner’s situational awareness in the. littoral, allowing the 
creation of a better mission plan. 

D. LIMITATIONS 

While this study does have merit because it was conducted on image analysts 
performing tasks pertinent to amphibious warfare, there were weaknesses associated with 
its design. Specifically, the data collected, specificity, and the lack of control are all noted 
as limitations. 

1. Data Collected 

One type of data that should have been specifically collected during this study 
pertained to the analyst’s relative certainty levels for different image types used to 
complete a particular subtask. While it is suspected that the user would still be most 
certain with results derived from high NIIRS panchromatic, this sort of data might have 
provided additional insight as to what in formats other than panchromatic makes the 
analyst uncertain. 

2. Specificity 

This study was focused on analysts supporting amphibious warfare. The results 
from this study were then extended to planners preparing amphibious missions. The 
benefits of exploring this single area in so much depth limits extension of the results to a 
larger unstudied population. 
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3 . 



Observer Control 



The author’s lack of control over the situation was a limitation: there was a great 
deal of variability associated with image quality. A more controlled study could have 
been created by using having the analysts all interpret the same littoral images, which 
would have reduced some of the variability in the study based on image type and quality. 
Analysts could have accessed these hypothetical images via and INTELINK web page at 
the Naval Postgraduate School. Once they accessed these electronic images, they 
analysts could perform their literal interpretation. 
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VI. RESULTS 



The first hypothesis — that there would be certain task elements for which the 
analyst would be less certain with their results — was validated. These task elements 
included: beach egress points determination, obstacle identification, surface characteristic 
determination, determining avenues of approach, underwater obstacle identification, and 
mobile orders of battle. 

The specific task elements that were not observed include: key terrain 
identification, terrain effects on supporting arms; general climatic description; underwater 
gradient in the vicinity of the landing site out to the three-fathom curve; average sea-state 
determination; mobility corridor determination; rail network determination; availability 
and condition of the civil/military telecommunications network; and avenue of approach 
determination for ground or fixed wing air forces. 

Table 5 shows what formats analysts used to complete a given task during the 
period of observation. More precise numbers cannot be given in the table because the 
specific number of images that an analyst used during the interview was not noted. 
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Task 


Panchromatic 


Radar 


Maps 


MSI 


IR 


Coastal Configuration 


X 


X 








Beach Egress Points 


X 










Obstacle Identification 


X 


X 








Cover and Concealment 
Determination 






X 


X 




Surface Characteristic 
Determination 


X 






X 




Determining Avenues of 
Approach 


X 




X 




X 


Tidal Condition 
Determination 


x 










Determining the Nature of 
the Surf 


X 










Wind Condition 
Determination 


X 










Determining Water Currents 


X 










Underwater Obstacle 
Identification 


X 




X 






Road Network 
Characteristics 


X 










Inland Waterways 
Characteristics 


X 










Static Order of Battle 


X 


X 






X 


Mobile Orders of Battle 


X 


X 






X 


TOTAL 


14 


3 


3 


2 


3 



Table 5. Counts of Image Formats Used by Analysts 
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A. GENERAL TOPOGRAPHIC DESCRIPTION 

1. Coastal Configuration 

To characterize the coastal configuration, the subject first located the large water 
mass by its darker color in either the panchromatic or radar image format. He or she then 
moved toward the land by identifying the breakers, which appear as long thin white lines 
on the dark background. After identifying the coastline, the subject selected portions of 
the beach for further study in order to determine if they met the criteria of a possible 
landing site. The factors impacting the selection at this point include the expected size of 
the landing force, the type of landing craft used in the operation, and the beach size and 
shape. The first two factors were facts that are available to the subject. Possible landing 
areas are classified initially by their shape— straight, concave, or convex —then by 
measuring their physical size, and finally determining the geographic coordinates for the 
beach center. The subjects did not identify any uncertainty regarding this task. 

2. Beach Egress Points 

With possible beaches initially identified, the subject used the panchromatic 
images to determine if that area has adequate points for the landing force to exit the 
landing area from enroute to the objective. Adequate egress point descriptions included 
information regarding location, exit type, beach relief, and soil composition. The subject 
detected potential exits from differences in tone, which are either lighter or darker than 
the background. Depending on the degree of randomness that an exit exhibited, the 
subject classified the exit type as either manmade or natural. Natural exits exhibited 
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better blending into the background while the manmade exits displayed geometric, 
repetitive patterns. The subject was less willing to describe relief data from a single 
image. Therefore, he or she also analyzed topographic maps to make these 
determinations. The narrowest point of each exit was measured to provide the planner a 
measure of how easily an exit could be sealed by the defender. The three beach-soil 
classifications an analyst was confident in making were sand, pebbles, and rocks, which 
is a determination based on image texture and pattern. The subject exhibited uncertainty 
in making a more refined classification. 

3. Obstacle Identification 

The subject located manmade obstacles that were not buried by their shape and 
repetitive placement on the ground in either the radar or panchromatic image. Obstacles 
whose construction material was metal were detected on radar, even if the obstacle was 
not detected in panchromatic. Buried objects such as mines could possibly be detected 
because of ground discoloration, but the subject stated he or she would be uncertain 
regarding detection if the time between object burial and imaging was too long. The 
subject located natural obstacles such as rocks, inland streams, etc. by understanding how 
they can influence ground force movement. Inland streams are a darker color than the 
background and have a natural shape. The subject did not describe how other natural 
obstacles were detected, nor did he or she express uncertainty categorizing these features. 
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4. 



Cover and Concealment Determination 



In order to determine cover and concealment effects, the subject used a 
topographic map of the area. However, maps are of varying quality and the older the data 
on the map, the more the subject relied on other formats such as MSI. Non-gray scale, 
color composite maps were created for the area of interest. For areas with great seasonal 
variation in foliage, the subject ensured that the source image was from the same season 
as the operation execution. The subject selected the bands to best describe the level that 
the terrain provides protection from enemy fire (cover) and concealment from enemy 
forces. There was no standard color scheme used. However, the subject emphasized that 
he or she must make the color representation simple enough to be able to explain it to the 
commander. The subject did not express uncertainty in making this determination. 

5. Surface Characteristic Determination 

Both panchromatic and MSI were used to characterize the soil composition. The 
subject needed to make cognitive decisions about soil composition, drainage, and relief. 

For the panchromatic format, the subject could only classify beach soil as either 
sand, pebbles, or rocks based on their texture and pattern. The subject did not comment 
on soil characterization other than near the beach. The subject made soil drainage 
characterizations based on color, because areas such as streams and water-saturated land 
appear darker in the scene. A computerized version of stereoscopy produced a qualitative 
description of vertical relief using two panchromatic images. The subject exhibited 
uncertainty in making specific soil characterizations using panchromatic. 
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The subject used MSI color composites to characterize soil composition. Beyond 
band selection, the subject did not quantify how this was performed. The subject did not 
quantify how precise a characterization he or she could make from the presented image 
for either the beach material or the soil composition of the area immediately behind the 
beach. The subject used digital terrain elevation data (DTED) and the elevation data on 
hardcopy charts to characterize surface slope. The subject did not express uncertainty 
performing this task using MSI in conjunction with the elevation data. 

6. Determining Avenues of Approach 

Subjects in the study performed two tasks that are relevant to determining the 
avenues of approach for rotary wing forces, locating power lines and determining suitable 
helicopter landing zone locations (HLZ). Using a topographical map, the subject located 
power lines by their symbol and also located the nearest prominent feature. Then the 
feature was located in an IR image and the subject performed a localized search for the 
first tower. The tower was located by either looking for the tower directly or by locating 
the shadow it projected. The subject re-centered the image at periodic intervals where he 
or she thought the next support tower would be located. The subject was not certain they 
could locate all the power lines in a scene. 

In determining potential HLZ’s, the subject scanned the panchromatic scene’s 
wide-open grassy areas such as a golf course or athletic field because they generally have 
a flat slope and fewer power lines surround them. The subject then determined the size 
and shape of the potential landing spots. He or she did this by examining the boundary 
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between the soil comprising the landing spot and surrounding features such as trees or 
fence line. The subject used an independent format such as a topographical map or 
DTED to determine the terrain slope of the landing area. The subject also recommended 
an approach path based on prominent navigation features such as buildings. After 
determining the size and shape of the actual landing area, the subject searched the landing 
area and the approach path for flight hazards such as power lines or lamp poles using the 
method described in the previous paragraph. Then the analyst looked for access points 
on the ground such as a road network. Finally, the subject performed an area search for 
air defense equipment, military units, and medical units. 

Dense urban areas and threat condition added uncertainty to locating adequate 
HLZ. In addition, the subject expressed uncertainty in locating all flight hazards such as 
power lines and poles. 

B. GENERAL HYDROGRAPHIC DESCRIPTION 

1. Tidal Condition Determination 

The subject estimated the tidal conditions by comparing panchromatic images at 
high and low tides. A measurement of the distance from the water’s edge at low tide to 
the point high water line was made. The high water line was identified by the shading 
difference between the darker water-saturated sand and lighter dry sand. A potential 
landing site could be discounted if that measured difference was too great, because the 
lack of cover and concealment place the landing force at a higher risk. The subject also 
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compared the two images looking for differences such as obstacles that might be covered 
at high tide. The subject did not express uncertainty regarding this task. 

2. Determining the Nature of the Surf 

The subject determined the nature of the surf by making two measurements on the 
panchromatic image. He or she first measured the distance from the water's edge to the 
point where the surf first breaks, then measured the angle that the waves strike the beach. 
These measurements provide an indication to planners of how dangerous the surf is to 
landing craft and personnel. The subject did not express uncertainty regarding this task. 

3. Wind Condition Determination 

The subject estimated the wind condition by analyzing wave action using 
panchromatic images. The subject estimated wind direction to be orthogonal to long, 
thin, white lines that characterized the waves. Larger waves also indicate stronger winds. 
The subject analyzed the waves more carefully to determine whether the wind was 
blowing on or off shore. If the leading edge of the wave is clean (i.e., no water spray) the 
wind direction was blowing to sea. Otherwise, the wind was blowing onto land. The 
subject did not express uncertainty regarding this task. 

4. Determining Water Currents 

The subject estimated water current by analyzing sediment deposit on the beach 
using panchromatic imagery. The sediment was identified in the image by the variation 
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in texture between it and the other beach materials at the water’s edge. If the sediment 
deposit appears evenly distributed across the beachfront, the current runs directly into the 
beach. If the sediment collection was uneven, then the current runs orthogonal to the area 
with more sediment. 

In addition, the subject used the position of manmade objects such as breakwater 
and jetty if available to judge the water current direction. The water current will hit the 
breakwater first and deflect into the jetty in order to keep the sediment outside a channel. 
The subject did not express uncertainty regarding this task. 

5. Underwater Obstacle Identification 

The subject stated that offshore obstacles could show up in panchromatic imagery 
if they are at a depth shallow enough to affect surface wave action. Using a low tide 
image, the subject scanned the wave pattern for line breaks away from shore. More 
random breaking patterns can indicate point obstacles such as rocks, while uniform 
breaking patterns indicate possible sandbars. The subject expressed uncertainty in 
locating deeper underwater obstacles because those obstacles that did not break the water 
surface or affect the wave action would not appear in the image. The subject stated he or 
she would also review hydrographic charts. In order to locate known underwater rock 
formations. 
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C. TRANSPORTATION FACILITIES DETERMINATION 

1. Road Network Characteristics 

The subject assessed the military utility of road networks by their type, width, and 
construction material. The subject first identified roads by their long narrow shape. The 
gray scale shade of the road in the panchromatic image is an indicator of the construction 
material: dirt roads blend into the background, concrete appears light gray, and asphalt 
appears very dark. The number of entrances and exits to the roadway per unit distance 
were enumerated. In addition, the subject examined the terrain relief surrounding the 
road for potential choke points such as tunnels and bridges. The subject did not express 
uncertainty regarding this task. 

2. Inland Waterways Characteristics 

The subject used gray scale shade and shape differences to identify inland 
waterways. The subject stated that streams and riverbeds were darker than the 
surrounding background in the panchromatic image. In addition, the subject stated that 
the topographical maps did not completely describe inland waterways. The subject 
attempted to classify the waterway as manmade or natural based on how random or 
repetitive the waterway appeared in the scene. The subject also measured width at 
various points. The subject did not express uncertainty regarding this task. 
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D. 



ENEMY MILITARY SITUATION DETERMINATION 



The techniques used by subjects on panchromatic, radar, and IR images for 
determining force strength and disposition varied between static and dynamic orders of 
battle. Static orders of battle include electronic, missile, air, naval (in-port), industrial, 
and ground forces in garrison. Dynamic or mobile orders of battle included any 
dispersed force such as deployed ground forces or forward air bases. The subjects 
stressed that knowledge about the sensor and the enemy as well as analyst experience 
aided the exploitation process. They also stated that shadows could help or hinder 
exploitation. Shadows were beneficial for seeing features too small to be captured by the 
sensor, but over-elongated shadows could distort information, possibly leading to 
misidentification. When discriminating between classes of objects, subjects would use 
key recognition features or length-to- width ratios as an aid. 

1. Static Order of Battle 

In previously identified sites, the place’s function was already known. In this 
instance, the subject was primarily interested in updating the body of knowledge about 
the site. The subject used existing knowledge about a site, either other images or reports 
based on prior interpretation, to aid in completing the task. Radar, IR, and panchromatic 
image formats all were exploited. As the subject scanned the site, he or she enumerated 
and classified different pieces of equipment. 

The subject interpreted the scene using a variety of techniques to aid in object 
recognition. Equipment’s general shape might provide clues as to the way it would 
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appear in a scene or the marks it might leave on the ground when it is removed. In 
addition, systems requiring extensive maintenance could have more support vehicles that 
could be detected on IR or radar. The subject also scanned images for known patterns 
based on the way certain systems were spatially arranged. More specifically, the subject 
looked for regular patterns on a random background. Ground shadows were also used to 
aid in identifying key recognition features. Finally, relative size between unidentified 
objects was used to help classify objects. The subject also added that high NIIRS 
panchromatic imagery was better than a map, especially in the urban areas. This was 
because the map could not adequately capture the complexity of the scene. The subject 
analyzed overgrowth to determine the frequency of usage for an unoccupied site. The 
subject did not express uncertainty regarding this task. 

2. Mobile Orders of Battle 

The subject stated that mobile orders of battle such as dispersed troops were more 
difficult to locate than their static equivalents. He or she attributed this to the relatively 
limited baseline knowledge that could aid in the search, sometimes only geographic 
coordinates. For this worst case scenario, the subject electronically plotted the 
coordinates on the image and began a wide area search. The subject scanned areas closer 
to lines of communication such as roads, trails, and rivers before searching other areas. 
The subject searched these areas for known geometric patterns that could indicate the 
function of a site. Terrain configuration significantly impacted the subjects’ ability to 



84 



identify units or equipment. For example areas with dense vegetation or significant 
terrain relief were particularly difficult to locate targets of interest. 

In cases such as a ground unit departing garrison, the subject first searched known 
training areas for the missing units. Confirming their presence in these areas could be an 
indication of routine operations. The subject also stated that some units modified their 
operations in the field, so he or she looked for these differences to locate units. If the 
subject successfully located one piece of equipment, he or she would search the area for 
other equipment associated with that unit. The subject also estimated unit size by 
enumerating objects that could be located, then taking the difference from what a unit 
doctrinally deploys. The subject expressed uncertainty regarding this task. 
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VII. DISCUSSION 



This thesis approaches the task of determining the potential impact of HSI in 
amphibious planning using the following process. The first section provides an 
explanation of how analysts and planners interpret planning aid formats using visual 
information processing theory. In addition, it offers a possible explanation for why the 
false color format is disturbing to planners and analysts. Having established this, a 
method for HSI exploitation and interpretation are developed. Then the work of Delgrand 
(1993), Fay (1995), Collins (1996), Bergman (1996), and Stuffle (1996) are briefly 
presented as examples of where HSI can provide valuable information in areas where the 
analyst is uncertain. Finally, after establishing how the interpreter acquires better 
information, particularly in areas of uncertainty, the effects of planning an amphibious 
mission with an improved knowledge base are evaluated. 

A. INFERENCES FROM RESULTS 

The objective of this section is to explain how image analysts cognitively process 
the different image formats using visual information processing. This is accomplished by 
developing a processing model that is based on the literature review and the results of the 
study. The model is then extended to include planners. From the model, a possible 
explanation for why false color formats are so disturbing to planners and analysts is 
provided. 
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1. Explaining Image Interpretation Using Visual Information Processing 

The model that is developed in this section takes the form of a sequence of events 
that the analyst attempts to complete in order to fulfill a subtask. Distinct variations in 
other formats such as radar and MSI are accounted for after establishing the initial model, 
because the initial model is developed exclusively for the panchromatic format. Finally, 

methods that the analyst employed for reducing any certainty are reviewed. 

/ 

a. Processing sequence 

To complete a subtask, image interpretation seemed to follow a sequence, 
which is shown in Figure 23. The basic sequence for the panchromatic format starts with 
attempting to establish perceptual constancy. Then the analyst must find a starting point 
in the image and establish a search method, Next, he or she locates, then refines raw 
pieces of information. It is probable that to accomplish a subtask, the analyst must 
complete these intermediate steps several times before accomplishing the subtask. 
Finally, the analyst combines pieces of information in higher-order processes in order to 
fulfill the subtask. The confidence the analyst has in the end product has an associated 
certainty level. 

Stimulus-based and knowledge-based processing are used by the analyst 
in varying capacities while completing this sequence. When the analyst is confident about 
the format being exploited and the task to be completed, it appears that the analyst 
employs knowledge not found in the stimulus. Therefore, top-down processing has a 
heavier emphasis. However, if the analyst was uncertain with the format or the task being 
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accomplished, he or she could not employ knowledge as effectively. In this case, 
stimulus-based processing had greater emphasis. 




Figure 23. Flow Chart for Completing Interpretation 



(1) Establishing Perceptual Constancy. Given that 
perceptual constancy can be considered a form of general knowledge, it is also the 
analysts’ first attempt at top-down processing. If the analyst can establish perceptual 
constancy, which means that he or she has successfully stabilized the appearance of the 
image in the domains of shape, size and color, then it is possible that the analyst can 
employ other forms of general knowledge. This reduces the emphasis on stimulus-based 
processing. However, if the analyst is unable to effectively establish perceptual 
constancy, then he or she must probably rely on more heavily on stimulus-based 
processing. 
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There are several factors that affect how quickly and confidently 
an analyst can establish perceptual constancy for a remotely sensed image. To start, how 
well the analyst understands the image format itself is an important part of establishing 
perceptual constancy in all three domains (shape, size, and color), because the image 
properties (scale, brightness, tone, contrast, and resolution) vary for each format. 
Developing perceptual constancy is probably easiest in the panchromatic format because 
the analyst is continually establishing perceptual constancy with energy from this 
wavelength band as part of everyday experience. While the perspective varies from that 
which the analyst comes into contact with outside of work, experience aids the analyst 
because the he or she grows more comfortable viewing images with an overhead 
perspective and a variety of look angles. This experience aids in establishing size and 
shape constancy. In addition, the good spatial resolution found in panchromatic imagery 
allows shapes to be portrayed accurately, which appears to make establishing shape 
constancy easier. The need to establish color constancy was bypassed in most cases 
because the panchromatic images are displayed in gray scale. Consequently, perceptual 
constancy might have been easier to establish on these images because there is one less 
domain (color) to be stabilized prior to commencing the next step. 

(2) Locating a Starting Point / Search Methods. Next, the 
analyst determines where in the scene to start searching for raw pieces of information. In 
cases where the initial site location was already known, the subject plotted the geographic 
coordinates electronically onto the image. When the site was not known, the subject 
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needed additional information to provide that focus. In most cases, a general geographic 
reference from other report types such as those from informants or electronic signals 
provided a geographical starting point. 

With the initial point established, the analyst relies on learned 
techniques such as following lines of communication (roads, waterways, power lines, 
etc.) to locate pieces of information. Using techniques such as these are an indication 
that the analyst is employing knowledge not found in the stimulus to aid in processing. 
This results in the analyst reducing the search space without having to search the entire 
image. However, if a technique such as this cannot be employed, the analyst must 
emphasize stimulus-based techniques. During the study, analysts expressed uncertainty 
in performing wide area searches that have little cueing data. 

(3) Locating Raw Pieces of Information. Identification of 
raw pieces of information is the first event in which the analyst draws something out of 
an image that an observer can also experience. The knowledge that the analyst uses at 
this point probably better fits the form of stimulus-based processing method such as 
feature theory. 

The analyst probably locates the raw pieces of information by 
using the elements of recognition: shape, size, tone, texture, pattern shadow, site, and 
association (Campbell, 1996). To help the analyst draw the information out of the image, 
there would be qualifiers on the elements of recognition. For example, in the subtask 
“determine coastal configuration,” the analyst must identify pieces of raw information 
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about the shape of a length of coastline. In this case, the analyst will use three different 
qualifiers to locate that particular length of coastline. He or she first finds the breakers by 
using specific instances of shape (long, thin lines) and color (white on a dark 
background). The specific instance of association (breakers are near the coast) are then 
used to determine the shape of the coast. 

(4) Refining Raw Information. This step is significant in this 
process for two reasons. First, the refinement of raw information requires knowledge not 
found in the image for successful completion. As a result, it is better characterized in the 
top-down paradigm. In addition, it is also probably the first place where feedback to 
earlier steps can occur. For example, an analyst cannot start to search along a line of 
communication until that line of communication is properly identified. 

To refine raw pieces of information, the analyst performs tasks that 
Campbell (1996) describes: classification, enumeration, mensuration, and delineation. 
Continuing the “coastal configuration” example, in order to generate a set of coastline 
areas suitable for landing, the analyst would measure portions of the “raw” coastline to 
determine which areas meet the minimum size criteria for landing a particular sized force. 

(5) Higher-Order Processing. The analyst accomplishes the 
subtasks in this scenario by combining the refined pieces of information with specialized 
knowledge. In the “coastline configuration” example, the set of potential sites is perhaps 
rated by evaluating their shape (straight, concave, or convex) with respect to how the 
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enemy can concentrate firepower against the landing force and the known enemy troop 
locations. In this instance, the specialized knowledge would be an understanding of how 
different coastline shapes and troop disposition impact the placement of anti-landing 
defenses. 



b. Interpretation Variations Based on Format 
With the basic sequence established for the panchromatic format, 
variations in the sequence can be examined for the radar, IR, and MSI formats. This 
must be done because the basic image properties (scale, brightness, tone, contrast, and 
resolution) vary for each format. 

(1) Radar. The major difference in the visual information 
processing sequence between radar and panchromatic appears to be in the refining raw 
pieces of information step. After the analyst selected a piece of raw information to 
interpret, he or she talked about how the sensor collected energy, explaining the geometry 
necessary to get an adequate return or how that geometry affected the analyst’s ability to 
identify key features. For example, one analyst was trying to classify a warship in a 
crowded port, but the ships’ rocking motion smeared the return from the target of interest. 
The subject identified the ship by first employing knowledge about that country’s naval 
order of battle; this allowed the analyst to reduce the number of possibilities to three or 
four ships. Then he or she identified specific features on the ship such as approximate 
length and number and position of masts and gun turrets on the ship. These pieces of 
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information were then cross-referenced to a recognition guide to properly identify that 
warship. This example highlights the possibility that the analyst’s difficulty in locating 
and refining raw pieces of information may be an indication that the analyst has to use 
specialized knowledge about the format sooner in the interpretation process. However, in 
other cases the analyst was not able to refine the information to the level of granularity 
that they could get from a good resolution panchromatic image (e.g., it was a transport 
aircraft, but he or she could not tell whether the aircraft was a C-130). 

(2) IR. There did not appear to be any significant variations in 
interpretation for analysts exploiting IR images. However, the author only observed three 
IR scenes, which all appeared very similar to panchromatic images. 

(3) MSI. One area where the author specifically tried to 
collect data was in the exploitation of MSI false color images. The primary question is 
whether the form of such a composite made interpretation difficult. In the two 
applications observed it was unclear whether the analyst established color constancy for 
the MSI format. The subject specifically stated that he or she was not trying to equate 
color to a particular object (e.g., blue water or green vegetation). Rather, the analyst was 
trying to identify potential regions (e.g., this purple shaded are is a void of vegetation). 
In other words, the raw pieces of information the subject collected were areas of a certain 
color. Information refinement in this case was accomplished by integrating overlaying 
DTED onto the false color composite and eliminating areas based on slope. The author is 
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reluctant to make further inferences about the visual information processing methods 
employed to interpret MSI given the extremely small sample size. 

c. Uncertainty 

When the subtask is complete, the analyst has a particular confidence level 
with the result of their interpretation. Some of the factors that affect this confidence level 
are the basic image properties such as resolution and contrast. When the analyst is less 
confident in a given subtask or with a image format, he or she is unable to apply the 
specialized knowledge of image interpretation. In this case, it is more probable that he or 
she will place a heavier emphasis on the stimulus. 

However, the end user still needs the results of the analysts. As a result, 
subjects had mechanisms available, such as NIIRS Rating, supervisory check, sensor 
revisit, and older images, to increase the level of confidence they had in their judgments. 
The analyst informs the end user of their confidence in most first phase exploitation 
reports by reporting the image’s NIIRS rating. The author is viewing the NIIRS rating as 
a level of confidence because the analysts should be able to draw more information from 
higher NIIRS images. In addition, another trained military analyst always reviewed the 
subject’s work for errors. This involved the second analyst reviewing the image and the 
report produced from the image. Frequent sensor revisit and reports from other 
intelligence sources improved the subject’s confidence in making judgment calls, 
particularly during radar exploitation. Finally, the subject used other sources such as 
older panchromatic images to improve confidence. 
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2. Visual Information Processing Employed in Planning 

The visual information processing model is also applicable to planners, although 
it requires a slight modification in scope. While the analyst focuses on lower level 
products (i.e., answering subtasks), the planner is more focused on combining the results 
of subtasks such as coastal shape, enemy defense location, and obstacles to develop a 
situational awareness about the battlespace that is the foundation for the mission plan. 
After developing their situational awareness, they can employ the knowledge in the areas 
of tactics and weapons employment to develop the actual mission plan. 

Planners probably have the skills to complete the basic visual processing 
sequence for the panchromatic and map formats. Panchromatic and map formats are 
singled out as “interpretable” to all planners, because the planner can easily understand 
the assessments from panchromatic with minimal explanation. In addition, the planner 
has received the necessary training to interpret maps as part of their military education. 

Some planners such as Aviators and Targeting Officers may have additional 
experience with the IR format, because they use IR sensors such as night vision goggles 
(NVG) or forward looking infrared (FLIR) while conducting the mission. However, 
based on the author’s experience, the Aviators’ mission is still planned from 
panchromatic images and maps. So while the Aviator is flying the mission, he or she is 
comparing NVG / FLIR data to information previously acquired by the panchromatic and 
map formats. 

However, other planners may have had even less experience with formats other 
than panchromatic and, as a result, be less likely to believe an analyst’s assessment. 
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Wollenbecker (1999) commented that gray scale IR negatives were given a commander 
at one point in his career instead of the actual IR image, because the format looked more 
like a “photograph.” The person’s ability to process the image format, which is not 
panchromatic, is negatively impacted because he or she is expecting a photograph. 

a. Why False-Color is Confusing 

To illustrate why false-color are confusing, the first phase terrain 
categorization map of the KERNEL BLITZ 1997 data set in Figure 24 is used (S1TAC, 
1997). It confuses the analyst or planner because colors and textures used to represent 
objects might be associated in long-term memory with other objects. In the original 
image, there is a legend that describes the terrain categorization. However, the legend 
alone is insufficient because it does not reduce the internal confusion created by the color 
schemes in the image. 

One example of this confusion is the region with the appearance of molten 
lava that represents ocean. If the author were trying to locate targets of interest such as 
waterborne obstacles, he would obviously look in the water region. However, when the 
author looks at the ocean in this scene, he perceives molten lava. 

The internal confusion between whether he is looking at molten lava or 
ocean continues throughout the processing sequence. Initial attempts at top-down 
processing probably fail because the image violates color constancy (e.g., the water looks 
like molten lava and the land looks like water). While the author is able to locate the 
coastline, developing an effective search pattern for the water is difficult because there 
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are many textures in this portion of the scene. Consequently, the author has difficulty 
locating raw (or refining) pieces of information because the scene does not permit him to 
employ his knowledge about how to identify waterborne obstacles (i.e., looking for 
breaks in smooth water or in wave patterns). Therefore, the author would report to the 
planner that he could not locate any obstacles in the water and that he is very uncertain of 
his result. 




Figure 24. Terrain Categorization of KERNEL BLITZ 1997 Data (SITAC, 1997) 



Therefore, based on the work of this thesis, a person’s (analyst or planner) 
lack of training in false-color formats such as MSI and HSI prevent the effective 
establishment of color constancy. Without color constancy in these formats, people 
cannot completely establish perceptual constancy. In addition, they also have a more 
difficult time employing knowledge that they would otherwise use in image exploitation. 
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Consequently, they cannot complete the rest of the visual processing sequence efficiently, 
because perceptual constancy is the first step in the sequence. If the planner perceives 
that stimulus-based processing takes too long, he or she might find a more trusted format, 
even if the information is less accurate. 

B. HSI EXPLOITATION 

With a possible reason for why false color products such as HSI are confusing to 
planners established, it is possible to create a way to present these products to planners 
that minimizes problems with color constancy and subsequently to develop a 
methodology for exploiting HSI. This can be accomplished by the following: literal 
interpretation, non-literal interpretation, and presentation to the planner. 

1. Literal Interpretation of HSI 

Panchromatic images are the preferred image format by planners and analysts. 
They can be created from the HSI data cube by generating true color composites from the 
HSI data cube, then converting the resulting image to black and white. Figure 15 
illustrates another method of producing a panchromatic-like image, which is the first 
principal component image of Morro Bay. Then the analyst exploits (or planner uses) 
these images using the same visual information processing sequence already employed 
on other panchromatic images. The raw or refined pieces of information derived from 
the data cube will build the context necessary to perform spectral interpretation on the 
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image. Therefore, HSI can provide both the analyst and the planner a baseline of 
information and products that they are most comfortable exploiting and using. 

2. Non-literal Processing of HSI 

With literal processing completed, the analyst can begin spectral interpretation. 
This involves applying a series of algorithms or transformations to the data cube and 
visually interpreting the results. Prior literal processing established the context for the 
scene. This allows the analyst to use the data set in such a way that they are refining 
pieces of raw information using non-literal means instead of drawing out raw pieces of 
information for the first time. This process was illustrated above for a littoral scene in 
Figures 17-22. To further illustrate HSI exploitation, a partial review of the Collins 
(1996) results from his analysis of the WESTERN RAINBOW data set. PCA and SAM 
are the techniques reviewed below. It is also important to highlight that the data set 
Collins analyzed extended into the thermal IR region. 

a. Principal Components Analysis (PCA) 

In general, the first PC image is the positive weighted average of all the 
bands analyzed. For a thermal sensor, this produces an output that appears similar to that 
produced by a forward-looking infrared (FLIR) sensor. One is shown in Figure 25. a 
(Collins, 1996). Planners such as Aviators and Targeting Officers who have previously 
worked with FLIR data should have the knowledge necessary to interpret the image. 
These people should be able to identify the man-made objects as well as potential 
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concealment areas in Figure 25. a even if the spatial resolution prohibits further 
refinement. 



b. Spectral Angle Mapping (SAM) 

A typical classification approach, illustrated in Figure 20, is the SAM 
technique. Application of this process to the thermal data produced the image shown in 
Figure 25.b. The information level of information in Figure 25.b is different from the 
simulated FLIR image in Figure 25. a. For example, spatial resolution and contrast in 
Figure 25. a are insufficient to classify the “X” shaped array in the lower half of the image 
as anything other than vehicles. However, the SAM classification tells the analyst or 
planner that the ‘X’ shaped array is a group of T-80 tanks. 

One problem with the target map is that if too narrow of an angle value is 
chosen or too few spectra are used for comparison, much of the scene can remain 
unclassified. This results in much of the context in the original image being lost as a by- 
product of the analysis, ultimately making the SAM image more difficult to visually 
process. Luckily, context can be regained in this image by overlaying the target map 
onto a panchromatic image, as show in Figure 25. c. The image was produced in 
Microsoft Image Composer 1.5 after scanning Collins’ base images into .GIF format. 
Overlaying is a standard feature in hyperspectral manipulation software such as 
Hypercube (Pazak, 1999). More importantly, the image in Figure 25. c provides better 
information than either Figures 25. a or 25.b do independently. In Figure 25.c the five 
blue boxes in a star pattern are an array of T-80 tanks, which are also not camouflaged or 
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concealed in the terrain, making them easier to identify. Based on the author's 
experience, it is speculated that planners might feel the same way. 

3. Presentation to the Planner 

An HSI-derived intelligence package that is delivered to the planner should 
include panchromatic images, post-algorithm images, and written descriptions. 
Panchromatic images allow the planner develop the baseline for the area. Post-algorithm 
images are used to supplement the information already derived from the panchromatic 
image. As the planner is unfamiliar with the color schemes used in hyperspectral 
formats, one-paragraph narratives of what the colors describe should be accompany each 
of the post algorithm images. The narratives allow the planner to effectively combine the 
general knowledge of the area that the panchromatic image provides with the specialized 
knowledge from false color HSI products in the planning process without extensive 
training in HSI. 

C. MATCHING HSI CERTAINTY WITH ANALYST UNCERTAINTY 

If analysts and planners have established methods to perform subtasks and they 
are confident in the results, then they are unlikely to accept or develop a different 
procedure using a different format to get the same result. However, they might be more 
willing to accept methods to gain intelligence in areas where they are currently uncertain. 
Mine detection, bathymetry, and target detection are three areas where spectral analysis 
shows promise. 
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b. SAM results 



a. Simulated FLIR 



c. SAM overlaid onto 
Simulated FLIR 
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Figure 25. Hyperspectral Results from WESTERN RAINBOW (from Collins, 1996) 
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1 . 



Mine Detection 



Delgrand (1993) successfully detected land mines buried up to 15 cm deep in a 
variety of environments using thermal IR data collected in the 5.0 and 10.0 pm region of 
the EM spectrum. Selected results are shown in Figure 26. The circles in each image 
mark the land mine locations in image c. The additional processing shown in images b 
and c highlight the differences in soil temperature to the point where the analyst is 
confident in declaring the circled spots in image c landmines. The same technique could 
be done with spectral thermal imagery such as SEBASS. There are some suggestions that 
visible imagery can detect recently disturbed earth as well (Bergman, 1996; Collins, 
1996). 




a. unprocessed scene b. warm soil c. tagged mines 

Figure 26. Buried Land Mine Detection Using Dual Band IR (From Delgrand, 1993) 



2. Bathymetry 

Stuffle (1996) produced bathymetric images out to the 10-meter curve of Secret 
Harbor, Lake Tahoe, using HYDICE data. Stuffle accomplished this by first producing 
the “mask” of the water’s edge and the bottom materials in the harbor using PCA and 
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band selection shown in Figure 27. a. Blue indicates sand, yellow indicates bright rock, 
red indicates dark rock, and black indicates land above water. Then using these known 
spectra, as well as approximated atmospheric effect and water characteristics, he 
generated the bottom contour map shown in Figure 27.b using the Bierwirth (1993) 
Algorithm. These images can provide the planner a significant amount of intelligence 
when compared to that on a chart of the same area shown in Figure 27.c. The planner 
now knows what the bottom is made of, the depth of the water in the pertinent areas, and 
the gradient of the seabed. This intelligence is used to determine where the most 
desirable points to land are, as well as what areas can damage landing craft. An analyst 
could not provide any of the intelligence to this level of detail by inspecting any existing 
format. 



3. Target Detection and Identification 

Bergman (1996) and Fay (1995) have analyzed HSI data sets in order to 
determine how effectively military targets could be detected and classified. Bergman 
(1996) showed it was possible to detect and discriminate between real and decoy vehicles 
in a natural grass environment, based on the analysis of HSI data from Operation 
FOREST RADIANCE I. Fay (1995) showed the feasibility in detecting tactical targets in 
the desert environment based on his analysis of HSI data from Operation DESERT 
RADIANCE I. The significance in their work is that they show that a properly trained 
analyst has another method besides ground resolution to detect and classify targets. 
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a. Bottom Surface Materials Mask b. Bottom Contour Map 




Figure 27. Bathymetry of Secret Harbor, Lake Tahoe (From Stuffle, 1996; NOAA, 1987) 

This section started by discussing how image analysts should approach HSI 
exploitation in order to effectively process the data cube. Then some of the information 
that can be derived from a HSI data cube was presented. This progression now permits 
the exploration of how HSI can impact amphibious planning. 

D. AMPHIBIOUS PLANNING 

In warfare, the attacker chooses the point of attack to which the defender must 
react. Better mission plans take advantage of attackers’ capabilities while placing the 
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defender at the greater disadvantage. Assuming that better quality information at the 
proper place and time produce a better mission plan, the question the rest of this chapter 
addresses is, “How does HSI influence the planning process such that the attackers’ 
strengths are optimized while placing the defender at the greatest disadvantage?” 

In the largest quantitative sense, this better information allows the attacker to 
chose the best landing site or point of attack. Better characterization of the battlespace 
can mean that the attacker is less likely to attack into the heart of the enemy’s defense, 
unless the objective and time constraints demand it. By moving the point of attack away 
from the defender’s strength, the attacker forces the defender to generate combat power at 
the point of attack in the same manner as the attacker. However, the defenders force 
must move over land, using a slower rate of advance than the attacker (Gatchel, 1996). 

Better characterization of the battlespace has other benefits. First, the planner can 
accurately estimate what force size is needed to accomplish the mission. In addition, if 
the analyst is able to detect defenses in a possible landing area that did not previously 
exist, the planning staff can deal with those contingencies before the execution phase of 
the operation. For example, if land or seaborne mines are detected near the operation 
using HSI, the planning staff can plan to sweep, detonate, or maneuver around the 
minefield. All these options place the defender at a greater disadvantage, because they 
must now compensate for the failure of these defenses during combat. 

Another option the attacker has regards the precision of his attack. If the analyst 
can more accurately characterize the area of operations by discriminating between decoy 
and real sites, the attacker’s force can be more efficiently employed because decoy sites 
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are ignored. Similarly, should the analyst be able to discriminate between targets such T- 
72 and T-80 tanks as Collins (1996) did, this allows the attacker to apply the proper 
amount of force to achieve a desired effect without wasting valuable munitions. Once 
again, the defender is placed at a disadvantage, because the attacker has not over- 
committed his strength to any particular target. 
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VIII. CONCLUSION / RECOMMENDATIONS 



In conclusion, this thesis argues that specialized knowledge in remote sensing and 
intelligence allows the analyst to establish perceptual constancy and develop the context 
required to apply knowledge not found in the stimuli to visual information processing. 
Furthermore, if the analyst is provided specialized knowledge about hyperspectral, he or 
she can produce additional intelligence that would otherwise that be unattainable, which 
results in the planner having better knowledge about the operating area from which to 
produce a mission plan from. 

Images derived from hyperspectral sensors are imagery intelligence. This format 
should be integrated as an operational capability as soon as possible. Hyperspectral 
sensors could be deployed with tactical forces in a manner similar to the F-14 Tactical 
Aviation Reconnaissance Pod System (TARPS). However, until image analysts are 
properly trained and can explain the results of their work to a mission planner, HSI will 
not gain widespread acceptance because of the complicated nature of the displayed 
results. In addition, further work in the areas of intelligence architecture and systems 
must be addressed to maximize the benefits of spectral imagery exploitation. HSI is 
another format that the analyst should have at his disposal to ensure the battlespace is 
properly described, so that the planner can create the mission that best meets the 
prescribed objectives. 



109 



THIS PAGE LEFT INTENTIONALLY BLANK 



110 



APPENDIX A 



COLOR FIGURES 
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Figure 2. 1:50,000 Topographical Map ofKontum, Vietnam (NIMA, 1995) 
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Figure 8. Images with Identical GSD and Varying Image Quality (IRARS, 1995) 
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a. True Color Bands 1, 2, 3 (BGR) 



b. NIR False Color TM Bands 2, 3, 4 (BGR) 



c. False Color TM Bands 5, 7, 6 (BGR) d. False Color TM Bands 4, 7, 1 (BGR) 

Figure 14. True and False Color Composite Images of Morro Bay, CA (Short, 1999) 
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PC 1 



PC 2 



PC 3 



PC 4 



PC 5 



PC 6 



PC 7 



PC Color Composite- 
images 1, 2, 3 (RGB) 

Figure 15. PC Component Images of Morro Bay, CA (Short, 1999) 
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Figure 16. Sample Hyperspectral Data Cube (From Short, 1999) 




Figure 17. True Color Composite of Eglin AFB (Thanks to Chris Simi, Night Vision Lab) 
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Figure 18. Mean Spectra for Regions of Interest 
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Figure 20. Target Map of Eglin AFB Produced by SAM 
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Figure22. SMF Color Composite with Selected SAM Data Overlaid 
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Figure 24. Terrain Categorization of KERNEL BLITZ 1997 Data (SITAC, 1997) 



118 







a. Simulated FLIR 



b. SAM results 



c. SAM overlaid onto 
Simulated FLIR 

Figure 25. Hyperspectral Results from WESTERN RAINBOW (from Collins, 1996) 
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a. unprocessed scene b. warm soil c. tagged mines 

Figure 26. Buried Land Mine Detection Using Dual Band IR (From Delgrand, 1993) 




a. Bottom Surface Materials Mask 



i Ov: 0 

b. Bottom Contour Map 



c. Chart of the Same Area 

Figure 27. Bathymetry of Secret Harbor, Lake Tahoe (From Stuffle, 1996; NOAA, 1987) 
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APPENDIX B: TASK ELEMENT LIST 



A. Determine the general topographic description for the area in question 

1 . Determine coastal configuration 

2. Determine egress routes of coastal areas 

3. Identify key terrain in the area 

4. Obstacle identification 

5. Determine the extent that the terrain provides cover and concealment 

6. Determine surface characteristics: slope, materials, and drainage 

7. Determine mobility corridors 

8. Determine avenues of approach: ground and air forces 

9. Determine how this terrain description affects supporting arms 

B. Determine the general hydrographic characteristics for the area in question 

1 . Average Sea-State Determination 

2. Determine the underwater gradient for the landing site from the waters 
edge to the 3-fathom curve 

3. Determine the nature of the surf IVO the beach 

4. Determine the tidal conditions 

5. Determine wind conditions 

6. Determine water currents: type, direction and speed 

7. Determine existence and location of underwater obstacles 

C. Determine the general climatic characteristics for the area in question 

D. Determine the nature and extent of transportation facilities in the area 

1 . Determine characteristics of road network 

2. Determine characteristics of the existing rail network: 

3. Determine characteristics of inland waterways 

4. Determine the availability and condition of existing civil / military 
telecommunications network 

E. Determine enemy military situation 

1 . Static Orders of Battle 

2. Mobile Orders of Battle 
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